* Re: [PATCH 7/9] http: discard hash in dumb-http http_object_request
From: Patrick Steinhardt @ 2026-07-03 11:27 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20260702080707.GG2029434@coredump.intra.peff.net>
On Thu, Jul 02, 2026 at 04:07:07AM -0400, Jeff King wrote:
> The flag handling could be removed if the hash-discard function were
> idempotent. This could be done easily-ish by having the underlying
> hash functions (like the ones in sha256/openssl.h) set the context
> pointer to NULL after free-ing. But it's something that every platform
> implementation would have to remember to do, and the benefit for the
> callers is not that huge (it would let us shave a few lines here and
> probably in a few other spots).
This answers an earlier question of mine. It would indeed be great if it
was idempotent -- I've been bitten by interfaces like this once too
much, where you have to be very careful to manage the lifetime of a
specific object. The prime example of this are (were? I don't quite
recall whether we fixed that interface) reference transactions, and that
caused a bunch of bugs in the past.
Patrick
^ permalink raw reply
* lipo availability [was: [PATCH v2] Makefile: link osxkeychain & support universal Rust]
From: Ben Knoble @ 2026-07-03 12:02 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: Shardul Natu via GitGitGadget, git, Kristoffer Haugsbakk, Shnatu,
Koji Nakamaru
In-Reply-To: <akZQmDYe9MtTdGM2@pks.im>
> Le 2 juil. 2026 à 07:57, Patrick Steinhardt <ps@pks.im> a écrit :
>
> On Wed, Jul 01, 2026 at 10:01:43PM +0000, Shardul Natu via GitGitGadget wrote:
>> From: Shnatu <snatu@google.com>
>>
>> Also, introduce native support for macOS Universal Binaries
>> (multi-architecture builds) in the Git build system by allowing
>> the user to specify a list of target triples in the RUST_TARGETS
>> environment variable.
>
>> 3. On macOS, if multiple targets are specified, we use lipo to
>> combine them into a single Universal static library at
>> target/release/libgitcore.a.
>>
>
>> diff --git a/Makefile b/Makefile
>> index 1f3f099f5c..8d49ecc897 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -3019,11 +3030,33 @@ scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)
>> $(LIB_FILE): $(LIB_OBJS)
>> $(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
>>
>> +ifndef NO_RUST
>> +ifeq ($(RUST_TARGETS),)
>> $(RUST_LIB): Cargo.toml $(RUST_SOURCES) $(LIB_FILE)
>> $(QUIET_CARGO)cargo build $(CARGO_ARGS)
>> +else
>> +ifneq ($(words $(RUST_TARGETS)),1)
>> +ifneq ($(uname_S),Darwin)
>> +$(error Building universal Rust libraries requires macOS (lipo is not available on $(uname_S)))
>> +endif
>> +endif
>> +
>> +RUST_MEMBER_LIBS = $(foreach target,$(RUST_TARGETS),target/$(target)/$(RUST_BUILD_CONFIG)/$(RUST_LIB_NAME))
>> +$(RUST_MEMBER_LIBS): target/%/$(RUST_BUILD_CONFIG)/$(RUST_LIB_NAME): Cargo.toml $(RUST_SOURCES) $(LIB_FILE)
>>
>> + $(QUIET_CARGO)cargo build $(CARGO_ARGS) --target $*
>> +
>> +$(RUST_LIB): $(RUST_MEMBER_LIBS)
>> + $(QUIET_GEN)\
>> + if [ $(words $(RUST_TARGETS)) -gt 1 ]; then \
>> + lipo -create $^ -output $@; \
>
> Can we assume lipo to be generally available on macOS?
From my digging, universal binaries to support the PowerPC transition to Intel have been available since Xcode 2.1 from 2005 (whose release notes, if they ever existed, have been impossible for me to find). Of course for modern (Universal Binary 2) formats, you will need newer Xcode: the format was announced in 2020, so I suspect Xcode 12 is a minimum but not necessarily sufficient version. I have not been able to find a release note specific to UB2, though in 12.2 there is mention of both universal binaries and the arm architecture. Apple’s announcement [1] supports the argument for v12.
[1]: https://www.apple.com/au/newsroom/2020/06/apple-announces-mac-transition-to-apple-silicon/
^ permalink raw reply
* Re: weird quadratic reftable behavior, was: Re: [PATCH 3/3] t5551: pack refs after creating many tags
From: Kristofer Karlsson @ 2026-07-03 12:09 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Jeff King, Michael Montalbo, git, Junio C Hamano
In-Reply-To: <akTm7BDohsy85sN8@pks.im>
On Wed, 1 Jul 2026 at 12:07, Patrick Steinhardt <ps@pks.im> wrote:
> >
> > I can send a proper patch if needed/wanted, but I might have missed
> > something silly here.
>
> Nice gains. I certainly think it would make sense to polish this a bit
> and then cast it into a patch.
>
> Patrick
I have a small draft here https://github.com/gitgitgadget/git/pull/2166
but I am honestly not sure if it's worth submitting as a patch - the
change is somewhat small, but spread out, and I failed to properly
reproduce the performance win in any realistic scenario (I had to
disable compaction to see the improvement).
I would want to rely on your expertise to know if this change
would be valuable to discuss as a patch at all.
Thanks,
Kristofer
^ permalink raw reply
* Programmatically edit the git rebase sequence?
From: Matthias Beyer @ 2026-07-03 12:02 UTC (permalink / raw)
To: git; +Cc: neikos
[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]
Hi git people,
in a recent conversation at work, the question of how to
programmatically edit the git-rebase sequence came up.
Example use case:
I have a branch that touches a number of files, adds some files and
removes some files.
When rebasing, I want to split all commits that touched a certain subset
of files, for the clearity of the history.
I look at the output of
git log master..mybranch --oneline --diff-filter=M -- "./subdir/*.rs"
to find all commits in that subdir that only touched the files. All of
these commits are to be "edit"ed.
Now I fire up `git rebase -i master` and manually(!) match the list from
above `git-log` call and find the respective commits to edit them.
Is there a way I am not aware of to do that manual step programatically?
Something like
git rebase -i master --edit-commits="$(git log master..mybranch --diff-filter=M --format="%H" -- "./subdir/*.rs")"
would be convenient here, although I would understand if that is too
much clutter for the already very heavy git CLI interface :-)
Maybe I am just not aware of the obvious solution - I would be happy to
learn that there is already one!
Best,
Matthias
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: Programmatically edit the git rebase sequence?
From: Michal Suchánek @ 2026-07-03 12:17 UTC (permalink / raw)
To: Matthias Beyer; +Cc: git, neikos
In-Reply-To: <akei64goQf3nFhX4@hikari>
On Fri, Jul 03, 2026 at 02:02:33PM +0200, Matthias Beyer wrote:
> Hi git people,
>
> in a recent conversation at work, the question of how to
> programmatically edit the git-rebase sequence came up.
>
> Example use case:
>
> I have a branch that touches a number of files, adds some files and
> removes some files.
> When rebasing, I want to split all commits that touched a certain subset
> of files, for the clearity of the history.
>
> I look at the output of
>
> git log master..mybranch --oneline --diff-filter=M -- "./subdir/*.rs"
>
> to find all commits in that subdir that only touched the files. All of
> these commits are to be "edit"ed.
>
> Now I fire up `git rebase -i master` and manually(!) match the list from
> above `git-log` call and find the respective commits to edit them.
>
> Is there a way I am not aware of to do that manual step programatically?
> Something like
>
> git rebase -i master --edit-commits="$(git log master..mybranch --diff-filter=M --format="%H" -- "./subdir/*.rs")"
>
> would be convenient here, although I would understand if that is too
> much clutter for the already very heavy git CLI interface :-)
>
> Maybe I am just not aware of the obvious solution - I would be happy to
> learn that there is already one!
Hello,
you could probably set a script as EDITOR (environment variable). Not
great for one-off scripts I guess.
Thanks
Michal
^ permalink raw reply
* Re: [PATCH v3 2/5] builtin/refs: add "delete" subcommand
From: Patrick Steinhardt @ 2026-07-03 12:38 UTC (permalink / raw)
To: Toon Claes; +Cc: git, Junio C Hamano
In-Reply-To: <87tsqg2vja.fsf@emacs.iotcl.com>
On Fri, Jul 03, 2026 at 12:54:17PM +0200, Toon Claes wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/builtin/refs.c b/builtin/refs.c
> > index f0faabf45a..edb7d61663 100644
> > --- a/builtin/refs.c
> > +++ b/builtin/refs.c
> > @@ -175,6 +178,52 @@ static int cmd_refs_optimize(int argc, const char **argv, const char *prefix,
> > return pack_refs_core(argc, argv, prefix, repo, refs_optimize_usage);
> > }
> >
> > +static int cmd_refs_delete(int argc, const char **argv, const char *prefix,
> > + struct repository *repo)
> > +{
> > + static char const * const refs_delete_usage[] = {
> > + REFS_DELETE_USAGE,
> > + NULL
> > + };
> > + const char *message = NULL;
> > + unsigned flags = 0;
> > + struct option opts[] = {
> > + OPT_STRING(0, "message", &message, N_("reason"),
> > + N_("reason of the update")),
> > + OPT_BIT(0 ,"no-deref", &flags,
> > + N_("update <refname> not the one it points to"),
> > + REF_NO_DEREF),
>
> Would it make sense to allow both --deref and --no-deref? (and --deref
> being the default)
Our "parse-options.h" subsystem is clever enough to make this work
already :)
> > diff --git a/t/t1464-refs-delete.sh b/t/t1464-refs-delete.sh
> > new file mode 100755
> > index 0000000000..efff7d0574
> > --- /dev/null
> > +++ b/t/t1464-refs-delete.sh
> > @@ -0,0 +1,130 @@
> > +#!/bin/sh
> > +
> > +test_description='git refs delete'
> > +
> > +. ./test-lib.sh
> > +
> > +setup_repo () {
> > + git init "$1" &&
> > + test_commit -C "$1" A &&
> > + test_commit -C "$1" B
> > +}
> > +
> > +test_expect_success 'delete without oldvalue verification' '
> > + test_when_finished "rm -rf repo" &&
> > + setup_repo repo &&
> > + A=$(git -C repo rev-parse A) &&
> > + git -C repo update-ref refs/heads/foo $A &&
> > + git -C repo refs delete refs/heads/foo &&
> > + test_must_fail git -C repo show-ref --verify -q refs/heads/foo
>
> Why not use `git refs exists` here? And why use `git -C repo` in this
> test, and `cd repo` in the other?
Yeah, there isn't really a good reason.
[snip]
> > +test_expect_success 'delete symref with --no-deref leaves target intact' '
> > + test_when_finished "rm -rf repo" &&
> > + setup_repo repo &&
> > + (
> > + cd repo &&
> > + A=$(git rev-parse A) &&
> > + git update-ref refs/heads/foo $A &&
> > + git symbolic-ref refs/heads/symref refs/heads/foo &&
> > + git refs delete --no-deref refs/heads/symref &&
> > + test_must_fail git refs exists refs/heads/symref &&
> > + git refs exists refs/heads/foo
> > + )
>
> What happens if you delete a symref and provide an <old-value>?
That's a good question indeed. It verifies that the symref target
resolves to <old-value>. That's the exact same behaviour as `git
update-ref -d`, even though it may be a bit on the funny side.
I've fixed up the above test and added another test for this behaviour,
but I'm not sure whether it makes sense to send another reroll for this.
Thanks!
Patrick
^ permalink raw reply
* Re: Unexpected recursion in 'git rm'
From: Patrick Steinhardt @ 2026-07-03 12:39 UTC (permalink / raw)
To: Евгений Плискин
Cc: git
In-Reply-To: <1756071445.20260703123414@gmail.com>
Adding the mailing list back into Cc.
On Fri, Jul 03, 2026 at 12:34:14PM +0300, Евгений Плискин wrote:
> > This is expected behaviour, as the argument to git-rm(1) is a pathspec, and "*" matches directory separators by default, see also gitglossary(7) under "pathspec":
> > • the pathspec up to the last slash represents a directory prefix. The scope of that pathspec is limited to that subtree.
> > • the rest of the pathspec is a pattern for the remainder of the pathname. Paths relative to the directory prefix will be matched against that pattern using fnmatch(3); in particular, * and ? can match directory separators.
> > For example, Documentation/*.jpg will match all .jpg files in the Documentation subtree, including Documentation/chapter_1/figure_1.jpg.
> > Could you maybe clarify which part of git-rm(1) made you think that this wouldn't happen?
>
> Thank you for your reply. I believe you are correct.
>
> I have made more research and found a way to remove files in current directory only without recursion into subdirectories:
> git rm -n ':(glob)*.json'
Yup, that wouldn't cross directory separators indeed.
Patrick
^ permalink raw reply
* [PATCH v3 00/12] reftable: harden against corrupted tables
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260624-pks-reftable-hardening-v1-0-66e4ce87c6b9@pks.im>
Hi,
this patch series addresses a bunch of errors that may happen when
trying to read corrupted tables. These errors include out-of-bounds
writes, out-of-bounds reads and the ability to hit abort(3p) calls.
The out-of-bounds write was originally reported by awo on the security
mailing list. As we never transfer reftables over the protocol it would
require local disk access to create such corrupted reftables, so there
isn't really an easy way to exploit these.
In any case, I took that chance and wrote a fuzzer for parsing the
tables, which surfaced a bunch of issues. At the end of this series
though the fuzzer can now run for an extended amount of time (2hrs+)
without surfacing any new issues.
Changes in v3:
- Make one of the unit tests a bit stricter.
- Link to v2: https://patch.msgid.link/20260629-pks-reftable-hardening-v2-0-b0228e7d908d@pks.im
Changes in v2:
- Introduce a test helper that writes a reftable block.
- Link to v1: https://patch.msgid.link/20260624-pks-reftable-hardening-v1-0-66e4ce87c6b9@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (12):
meson: support building fuzzers with libFuzzer
oss-fuzz: add fuzzer for parsing reftables
reftable/basics: fix OOB read on binary search of empty range
reftable/record: don't abort when decoding invalid ref value type
t/unit-tests: introduce test helper to write reftable blocks
reftable/block: fix OOB write with bogus inflated log size
reftable/block: fix OOB read with bogus block size
reftable/block: fix OOB read with bogus restart count
reftable/block: fix use of uninitialized memory when binsearch fails
reftable/block: fix OOB read with bogus restart offset
reftable/table: fix NULL pointer access when seeking to bogus offsets
reftable/table: fix OOB read on truncated table
Makefile | 1 +
ci/run-build-and-minimal-fuzzers.sh | 1 +
meson.build | 15 +++
meson_options.txt | 2 +
oss-fuzz/.gitignore | 1 +
oss-fuzz/fuzz-reftable.c | 74 ++++++++++++++
oss-fuzz/meson.build | 2 +
reftable/basics.c | 3 +
reftable/block.c | 39 +++++++-
reftable/record.c | 6 +-
reftable/table.c | 7 ++
t/unit-tests/u-reftable-basics.c | 11 +++
t/unit-tests/u-reftable-block.c | 189 ++++++++++++++++++++++++++++++++----
t/unit-tests/u-reftable-record.c | 24 +++++
t/unit-tests/u-reftable-table.c | 91 +++++++++++++++++
15 files changed, 440 insertions(+), 26 deletions(-)
Range-diff versus v2:
1: cd28489ce9 = 1: 53673a5044 meson: support building fuzzers with libFuzzer
2: 2f304e5bef = 2: 06037f8cc6 oss-fuzz: add fuzzer for parsing reftables
3: d74a5b244d = 3: 21b4025e1d reftable/basics: fix OOB read on binary search of empty range
4: f710410b28 = 4: dfed39b876 reftable/record: don't abort when decoding invalid ref value type
5: 0f90c22ccf = 5: badbfdc070 t/unit-tests: introduce test helper to write reftable blocks
6: b69a8c40a0 = 6: b0d13e79c5 reftable/block: fix OOB write with bogus inflated log size
7: 19046dbb2e ! 7: 612df3c909 reftable/block: fix OOB read with bogus block size
@@ Commit message
block size that exceeds the data we have actually read this can lead to
an out-of-bounds read:
- ==1458284==ERROR: AddressSanitizer: SEGV on unknown address 0x7d8ff7de4b7d (pc 0x55555598c339 bp 0x7fffffff4ef0 sp 0x7fffffff4eb0 T0)
- ==1458284==The signal is caused by a READ memory access.
- #0 0x55555598c339 in reftable_get_be16 ./build/../reftable/basics.h:118:9
- #1 0x55555598bee2 in reftable_block_init ./build/../reftable/block.c:344:18
- #2 0x555555813e0e in test_reftable_block__corrupt_block_size ./build/../t/unit-tests/u-reftable-block.c:540:8
- #3 0x5555557f684e in clar_run_test ./build/../t/unit-tests/clar/clar.c:335:3
- #4 0x5555557f2e69 in clar_run_suite ./build/../t/unit-tests/clar/clar.c:431:3
- #5 0x5555557f2882 in clar_test_run ./build/../t/unit-tests/clar/clar.c:636:4
- #6 0x5555557f375f in clar_test ./build/../t/unit-tests/clar/clar.c:687:11
- #7 0x5555557fa49d in cmd_main ./build/../t/unit-tests/unit-test.c:62:8
- #8 0x55555584b55a in main ./build/../common-main.c:9:11
- #9 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
- #10 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
- #11 0x555555694c24 in _start (./build/t/unit-tests+0x140c24)
+ ==2274138==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7c3ff6de2e3f at pc 0x55555598c6ea bp 0x7fffffff4ee0 sp 0x7fffffff4ed8
+ READ of size 1 at 0x7c3ff6de2e3f thread T0
+ #0 0x55555598c6e9 in reftable_get_be16 /home/pks/Development/git/build/../reftable/basics.h:119:20
+ #1 0x55555598c252 in reftable_block_init /home/pks/Development/git/build/../reftable/block.c:343:18
+ #2 0x555555813c70 in test_reftable_block__corrupt_block_size /home/pks/Development/git/build/../t/unit-tests/u-reftable-block.c:531:20
+ #3 0x5555557f684e in clar_run_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:335:3
+ #4 0x5555557f2e69 in clar_run_suite /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:431:3
+ #5 0x5555557f2882 in clar_test_run /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:636:4
+ #6 0x5555557f375f in clar_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:687:11
+ #7 0x5555557fa49d in cmd_main /home/pks/Development/git/build/../t/unit-tests/unit-test.c:62:8
+ #8 0x55555584b8aa in main /home/pks/Development/git/build/../common-main.c:9:11
+ #9 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b284) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
+ #10 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b337) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
+ #11 0x555555694c24 in _start (/home/pks/Development/git/build/t/unit-tests+0x140c24)
- ==1458284==Register values:
- rax = 0x00007d8ff7de4b7d rbx = 0x00007fffffff4f00 rcx = 0x0000000000000006 rdx = 0x0000000000000010
- rdi = 0x00007d8ff7de4b7d rsi = 0x00007bfff5cf0420 rbp = 0x00007fffffff4ef0 rsp = 0x00007fffffff4eb0
- r8 = 0x00000f807eb960b8 r9 = 0x0000000000000001 r10 = 0x00007bfff5cf05e7 r11 = 0x000000000000000f
- r12 = 0x00007fffffff58f8 r13 = 0x0000000000000001 r14 = 0x0000555555ee8160 r15 = 0x0000000000000000
- AddressSanitizer can not provide additional info.
+ 0x7c3ff6de2e3f is located 0 bytes after 47-byte region [0x7c3ff6de2e10,0x7c3ff6de2e3f)
+ allocated by thread T0 here:
+ #0 0x55555579e95b in malloc (/home/pks/Development/git/build/t/unit-tests+0x24a95b)
+ #1 0x5555559871c2 in reftable_malloc /home/pks/Development/git/build/../reftable/basics.c:24:9
+ #2 0x5555559872e8 in reftable_calloc /home/pks/Development/git/build/../reftable/basics.c:54:6
+ #3 0x55555598f0d3 in reftable_buf_read_data /home/pks/Development/git/build/../reftable/blocksource.c:67:2
+ #4 0x55555598ea7e in block_source_read_data /home/pks/Development/git/build/../reftable/blocksource.c:41:19
+ #5 0x55555598c555 in read_block /home/pks/Development/git/build/../reftable/block.c:224:9
+ #6 0x55555598b69e in reftable_block_init /home/pks/Development/git/build/../reftable/block.c:258:9
+ #7 0x555555813c70 in test_reftable_block__corrupt_block_size /home/pks/Development/git/build/../t/unit-tests/u-reftable-block.c:531:20
+ #8 0x5555557f684e in clar_run_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:335:3
+ #9 0x5555557f2e69 in clar_run_suite /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:431:3
+ #10 0x5555557f2882 in clar_test_run /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:636:4
+ #11 0x5555557f375f in clar_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:687:11
+ #12 0x5555557fa49d in cmd_main /home/pks/Development/git/build/../t/unit-tests/unit-test.c:62:8
+ #13 0x55555584b8aa in main /home/pks/Development/git/build/../common-main.c:9:11
+ #14 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b284) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
+ #15 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b337) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
+ #16 0x555555694c24 in _start (/home/pks/Development/git/build/t/unit-tests+0x140c24)
+
+ SUMMARY: AddressSanitizer: heap-buffer-overflow /home/pks/Development/git/build/../reftable/basics.h:119:20 in reftable_get_be16
+ Shadow bytes around the buggy address:
+ 0x7c3ff6de2b80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
+ 0x7c3ff6de2c00: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
+ 0x7c3ff6de2c80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
+ 0x7c3ff6de2d00: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
+ 0x7c3ff6de2d80: fa fa 00 00 00 00 00 00 fa fa fd fd fd fd fd fd
+ =>0x7c3ff6de2e00: fa fa 00 00 00 00 00[07]fa fa fa fa fa fa fa fa
+ 0x7c3ff6de2e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
+ 0x7c3ff6de2f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
+ 0x7c3ff6de2f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
+ 0x7c3ff6de3000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
+ 0x7c3ff6de3080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
+ Shadow byte legend (one shadow byte represents 8 application bytes):
+ Addressable: 00
+ Partially addressable: 01 02 03 04 05 06 07
+ Heap left redzone: fa
+ Freed heap region: fd
+ Stack left redzone: f1
+ Stack mid redzone: f2
+ Stack right redzone: f3
+ Stack after return: f5
+ Stack use after scope: f8
+ Global redzone: f9
+ Global init order: f6
+ Poisoned by user: f7
+ Container overflow: fc
+ Array cookie: ac
+ Intra object redzone: bb
+ ASan internal: fe
+ Left alloca redzone: ca
+ Right alloca redzone: cb
Verify that the claimed block size fits into the block data before using
it.
@@ t/unit-tests/u-reftable-block.c: void test_reftable_block__corrupt_log_block_siz
+ };
+ struct reftable_block block = { 0 };
+ struct reftable_buf data = REFTABLE_BUF_INIT;
++ uint32_t block_size;
++ unsigned char *p;
+
+ cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_REF, &rec, 1);
+
@@ t/unit-tests/u-reftable-block.c: void test_reftable_block__corrupt_log_block_siz
+ * the restart count and restart table relative to such a bogus block
+ * size must not access out-of-bounds memory.
+ */
-+ reftable_put_be24((uint8_t *) data.buf + 1, 0xffffff);
++ p = (unsigned char *) data.buf + 1;
++ block_size = reftable_get_be24(p);
++ cl_assert_equal_i(block_size, 47);
++ reftable_put_be24(p, block_size + 1);
+
+ block_source_from_buf(&source, &data);
+ cl_assert_equal_i(reftable_block_init(&block, &source, 0, 0, data.len,
8: bc0e0e5f30 = 8: 6df5cc741a reftable/block: fix OOB read with bogus restart count
9: 668108e851 = 9: 74e7d35bd9 reftable/block: fix use of uninitialized memory when binsearch fails
10: bb3f76e084 = 10: 3264933eba reftable/block: fix OOB read with bogus restart offset
11: 6b7e61728b = 11: 638964395c reftable/table: fix NULL pointer access when seeking to bogus offsets
12: ea051207e1 = 12: 6745a7fca8 reftable/table: fix OOB read on truncated table
---
base-commit: ab776a62a78576513ee121424adb19597fbb7613
change-id: 20260623-pks-reftable-hardening-f54de69fea63
^ permalink raw reply
* [PATCH v3 01/12] meson: support building fuzzers with libFuzzer
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
To support fuzzing via libFuzzer one has to pass a couple of compiler
options:
- It is mandatory to enable the "fuzzer-no-link" sanitizer for
coverage feedback.
- It is recommended to enable at least one more sanitizer to catch
issues, like the "address" sanitizer.
- The fuzzing executables need to be linked with "-fsanitize=fuzzer"
to wire up libFuzzer itself.
The first two items can already be achieved via the "-Db_sanitize="
option. But the last item cannot easily be achieved, as we can only
configure global link arguments.
Introduce a new "-Dfuzzers_link_args=" build option to plug this gap.
Add documentation so that users know how to set up libFuzzer.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
meson.build | 15 +++++++++++++++
meson_options.txt | 2 ++
oss-fuzz/meson.build | 1 +
3 files changed, 18 insertions(+)
diff --git a/meson.build b/meson.build
index 3247697f74..9df6fbb0a5 100644
--- a/meson.build
+++ b/meson.build
@@ -161,6 +161,21 @@
# These machine files can be passed to `meson setup` via the `--native-file`
# option.
#
+# Fuzzing
+# =======
+#
+# Meson supports building the fuzzing targets by setting `-Dfuzzers=true`. By
+# default, the targets will be built without libFuzzer and thus won't be usable
+# for fuzzing. You have to configure a couple of options to properly wire up
+# libFuzzer:
+#
+# $ meson setup build-fuzzers \
+# -Db_sanitize=address,fuzzer-no-link \
+# -Dfuzzers=true \
+# -Dfuzzers_link_args=-fsanitize=fuzzer
+# $ meson compile -C build-fuzzers
+# $ ./build-fuzzers/oss-fuzz/fuzz-config <args>
+#
# Cross compilation
# =================
#
diff --git a/meson_options.txt b/meson_options.txt
index d936ada098..dc88f130d7 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -131,3 +131,5 @@ option('test_utf8_locale', type: 'string',
description: 'Name of a UTF-8 locale used for testing.')
option('fuzzers', type: 'boolean', value: false,
description: 'Enable building fuzzers.')
+option('fuzzers_link_args', type: 'array', value: [],
+ description: 'Linker arguments used to link fuzzers. Use -fsanitize=fuzzer for fuzzing.')
diff --git a/oss-fuzz/meson.build b/oss-fuzz/meson.build
index 878afd8426..10bcac2f6d 100644
--- a/oss-fuzz/meson.build
+++ b/oss-fuzz/meson.build
@@ -16,5 +16,6 @@ foreach fuzz_program : fuzz_programs
fuzz_program,
],
dependencies: [libgit_commonmain],
+ link_args: get_option('fuzzers_link_args'),
)
endforeach
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 02/12] oss-fuzz: add fuzzer for parsing reftables
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
Add a new fuzzer that exercises our parsing of reftables. Fallout from
this fuzzer will be fixed over subsequent commits.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Makefile | 1 +
ci/run-build-and-minimal-fuzzers.sh | 1 +
oss-fuzz/.gitignore | 1 +
oss-fuzz/fuzz-reftable.c | 74 +++++++++++++++++++++++++++++++++++++
oss-fuzz/meson.build | 1 +
5 files changed, 78 insertions(+)
diff --git a/Makefile b/Makefile
index 1cec251f43..89d3edd5ea 100644
--- a/Makefile
+++ b/Makefile
@@ -2599,6 +2599,7 @@ FUZZ_OBJS += oss-fuzz/fuzz-date.o
FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
FUZZ_OBJS += oss-fuzz/fuzz-parse-attr-line.o
+FUZZ_OBJS += oss-fuzz/fuzz-reftable.o
FUZZ_OBJS += oss-fuzz/fuzz-url-decode-mem.o
.PHONY: fuzz-objs
fuzz-objs: $(FUZZ_OBJS)
diff --git a/ci/run-build-and-minimal-fuzzers.sh b/ci/run-build-and-minimal-fuzzers.sh
index e7b97952e7..37b24b092d 100755
--- a/ci/run-build-and-minimal-fuzzers.sh
+++ b/ci/run-build-and-minimal-fuzzers.sh
@@ -21,6 +21,7 @@ date
pack-headers
pack-idx
parse-attr-line
+reftable
url-decode-mem
"
diff --git a/oss-fuzz/.gitignore b/oss-fuzz/.gitignore
index f2d74de457..dc7a127a62 100644
--- a/oss-fuzz/.gitignore
+++ b/oss-fuzz/.gitignore
@@ -5,4 +5,5 @@ fuzz-date
fuzz-pack-headers
fuzz-pack-idx
fuzz-parse-attr-line
+fuzz-reftable
fuzz-url-decode-mem
diff --git a/oss-fuzz/fuzz-reftable.c b/oss-fuzz/fuzz-reftable.c
new file mode 100644
index 0000000000..c46eac2c6b
--- /dev/null
+++ b/oss-fuzz/fuzz-reftable.c
@@ -0,0 +1,74 @@
+#include "git-compat-util.h"
+#include "reftable/basics.h"
+#include "reftable/blocksource.h"
+#include "reftable/reftable-blocksource.h"
+#include "reftable/reftable-error.h"
+#include "reftable/reftable-iterator.h"
+#include "reftable/reftable-record.h"
+#include "reftable/reftable-table.h"
+#include "reftable/reftable-writer.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+ struct reftable_block_source source = { 0 };
+ struct reftable_buf buf = REFTABLE_BUF_INIT;
+ struct reftable_table *table = NULL;
+ int err;
+
+ if (reftable_buf_add(&buf, (const char *)data, size) < 0)
+ goto out;
+ block_source_from_buf(&source, &buf);
+
+ err = reftable_table_new(&table, &source, "fuzz-input");
+ if (err < 0)
+ goto out;
+
+ /*
+ * Exercise the ref, log and raw block iterators so that we cover as
+ * much of the parsing code as possible.
+ */
+ {
+ struct reftable_ref_record ref = { 0 };
+ struct reftable_iterator it = { 0 };
+
+ reftable_table_init_ref_iterator(table, &it);
+ if (!reftable_iterator_seek_ref(&it, ""))
+ while (!reftable_iterator_next_ref(&it, &ref))
+ ;
+
+ reftable_ref_record_release(&ref);
+ reftable_iterator_destroy(&it);
+ }
+
+ {
+ struct reftable_log_record log = { 0 };
+ struct reftable_iterator it = { 0 };
+
+ reftable_table_init_log_iterator(table, &it);
+ if (!reftable_iterator_seek_log(&it, ""))
+ while (!reftable_iterator_next_log(&it, &log))
+ ;
+
+ reftable_log_record_release(&log);
+ reftable_iterator_destroy(&it);
+ }
+
+ {
+ struct reftable_table_iterator it = { 0 };
+ const struct reftable_block *block;
+
+ if (!reftable_table_iterator_init(&it, table))
+ while (!reftable_table_iterator_next(&it, &block))
+ ;
+
+ reftable_table_iterator_release(&it);
+ }
+
+out:
+ if (table)
+ reftable_table_decref(table);
+ reftable_buf_release(&buf);
+ return 0;
+}
diff --git a/oss-fuzz/meson.build b/oss-fuzz/meson.build
index 10bcac2f6d..5a3854256b 100644
--- a/oss-fuzz/meson.build
+++ b/oss-fuzz/meson.build
@@ -6,6 +6,7 @@ fuzz_programs = [
'fuzz-pack-headers.c',
'fuzz-pack-idx.c',
'fuzz-parse-attr-line.c',
+ 'fuzz-reftable.c',
'fuzz-url-decode-mem.c',
]
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 03/12] reftable/basics: fix OOB read on binary search of empty range
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
`binsearch()` performs a binary search over a range of `sz` elements by
repeatedly calling the comparison function with indices into that range.
When the range is empty though, there is no valid index to call the
comparison function with. We still end up executing the comparison
function though with an index of 0, which of course will cause an
out-of-bounds read.
Return early when the range is empty.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/basics.c | 3 +++
t/unit-tests/u-reftable-basics.c | 11 +++++++++++
2 files changed, 14 insertions(+)
diff --git a/reftable/basics.c b/reftable/basics.c
index e969927b61..f0442a46cf 100644
--- a/reftable/basics.c
+++ b/reftable/basics.c
@@ -152,6 +152,9 @@ size_t binsearch(size_t sz, int (*f)(size_t k, void *args), void *args)
size_t lo = 0;
size_t hi = sz;
+ if (!sz)
+ return 0;
+
/* Invariants:
*
* (hi == sz) || f(hi) == true
diff --git a/t/unit-tests/u-reftable-basics.c b/t/unit-tests/u-reftable-basics.c
index 73566ed0eb..c5d83b6714 100644
--- a/t/unit-tests/u-reftable-basics.c
+++ b/t/unit-tests/u-reftable-basics.c
@@ -60,6 +60,17 @@ void test_reftable_basics__binsearch(void)
}
}
+static int unreachable_lesseq(size_t i UNUSED, void *args UNUSED)
+{
+ cl_fail("comparison function called for empty range");
+ return 0;
+}
+
+void test_reftable_basics__binsearch_empty(void)
+{
+ cl_assert_equal_i(binsearch(0, &unreachable_lesseq, NULL), 0);
+}
+
void test_reftable_basics__names_length(void)
{
const char *a[] = { "a", "b", NULL };
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 04/12] reftable/record: don't abort when decoding invalid ref value type
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
When decoding a ref record we read its value type from the block. In
case the type itself is invalid we call `abort()`. This is rather
heavy-handed though: the data we're reading is untrusted, so we should
treat the issue as a normal and not as a programming error.
Fix this by handling the error gracefully. Note that this also requires
us to set the value type later, as otherwise we might store an invalid
type in the record.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/record.c | 6 +++---
t/unit-tests/u-reftable-record.c | 24 ++++++++++++++++++++++++
2 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/reftable/record.c b/reftable/record.c
index fcd387ba5d..1fce441930 100644
--- a/reftable/record.c
+++ b/reftable/record.c
@@ -388,7 +388,6 @@ static int reftable_ref_record_decode(void *rec, struct reftable_buf key,
r->refname[key.len] = 0;
r->update_index = update_index;
- r->value_type = val_type;
switch (val_type) {
case REFTABLE_REF_VAL1:
if (in.len < hash_size) {
@@ -426,9 +425,10 @@ static int reftable_ref_record_decode(void *rec, struct reftable_buf key,
case REFTABLE_REF_DELETION:
break;
default:
- abort();
- break;
+ err = REFTABLE_FORMAT_ERROR;
+ goto done;
}
+ r->value_type = val_type;
return start.len - in.len;
diff --git a/t/unit-tests/u-reftable-record.c b/t/unit-tests/u-reftable-record.c
index 1bf2e170dc..9c95083ef4 100644
--- a/t/unit-tests/u-reftable-record.c
+++ b/t/unit-tests/u-reftable-record.c
@@ -11,6 +11,7 @@
#include "reftable/basics.h"
#include "reftable/constants.h"
#include "reftable/record.h"
+#include "reftable/reftable-error.h"
static void t_copy(struct reftable_record *rec)
{
@@ -202,6 +203,29 @@ void test_reftable_record__ref_record_roundtrip(void)
reftable_buf_release(&scratch);
}
+void test_reftable_record__ref_record_decode_invalid_value_type(void)
+{
+ struct reftable_buf scratch = REFTABLE_BUF_INIT;
+ struct reftable_record out = {
+ .type = REFTABLE_BLOCK_TYPE_REF,
+ };
+ struct reftable_buf key = REFTABLE_BUF_INIT;
+ uint8_t buffer[1024] = { 0 };
+ struct string_view dest = {
+ .buf = buffer,
+ .len = sizeof(buffer),
+ };
+
+ cl_must_pass(reftable_buf_addstr(&key, "refs/heads/master"));
+ cl_assert_equal_i(reftable_record_decode(&out, key, REFTABLE_NR_REF_VALUETYPES,
+ dest, REFTABLE_HASH_SIZE_SHA1, &scratch),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_record_release(&out);
+ reftable_buf_release(&key);
+ reftable_buf_release(&scratch);
+}
+
void test_reftable_record__log_record_comparison(void)
{
struct reftable_record in[3] = {
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 05/12] t/unit-tests: introduce test helper to write reftable blocks
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
Introduce a new test helper that allows us to write reftable blocks.
This helper will be used by subsequent commits.
Suggested-by: Christian Couder <christian.couder@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/unit-tests/u-reftable-block.c | 47 ++++++++++++++++++++++++-----------------
1 file changed, 28 insertions(+), 19 deletions(-)
diff --git a/t/unit-tests/u-reftable-block.c b/t/unit-tests/u-reftable-block.c
index f4bded7d26..f4e926ce3a 100644
--- a/t/unit-tests/u-reftable-block.c
+++ b/t/unit-tests/u-reftable-block.c
@@ -14,6 +14,31 @@ license that can be found in the LICENSE file or at
#include "reftable/reftable-error.h"
#include "strbuf.h"
+static int cl_reftable_write_block(struct reftable_buf *buf,
+ uint8_t block_type,
+ struct reftable_record *recs,
+ size_t nrecs)
+{
+ struct block_writer writer = {
+ .last_key = REFTABLE_BUF_INIT,
+ };
+ uint8_t block[1024];
+ int block_end;
+
+ cl_must_pass(block_writer_init(&writer, block_type, block, 1024,
+ 0, hash_size(REFTABLE_HASH_SHA1)));
+ for (size_t i = 0; i < nrecs; i++)
+ cl_must_pass(block_writer_add(&writer, &recs[i]));
+
+ block_end = block_writer_finish(&writer);
+ cl_assert(block_end > 0);
+
+ cl_must_pass(reftable_buf_add(buf, block, block_end));
+
+ block_writer_release(&writer);
+ return block_end;
+}
+
void test_reftable_block__read_write(void)
{
const int header_off = 21; /* random */
@@ -381,25 +406,13 @@ void test_reftable_block__ref_read_write(void)
void test_reftable_block__iterator(void)
{
struct reftable_block_source source = { 0 };
- struct block_writer writer = {
- .last_key = REFTABLE_BUF_INIT,
- };
struct reftable_record expected_refs[20];
struct reftable_ref_record ref = { 0 };
struct reftable_iterator it = { 0 };
struct reftable_block block = { 0 };
- struct reftable_buf data;
+ struct reftable_buf data = REFTABLE_BUF_INIT;
int err;
- data.len = 1024;
- REFTABLE_CALLOC_ARRAY(data.buf, data.len);
- cl_assert(data.buf != NULL);
-
- err = block_writer_init(&writer, REFTABLE_BLOCK_TYPE_REF,
- (uint8_t *) data.buf, data.len,
- 0, hash_size(REFTABLE_HASH_SHA1));
- cl_assert(!err);
-
for (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) {
expected_refs[i] = (struct reftable_record) {
.type = REFTABLE_BLOCK_TYPE_REF,
@@ -409,13 +422,10 @@ void test_reftable_block__iterator(void)
},
};
memset(expected_refs[i].u.ref.value.val1, i, REFTABLE_HASH_SIZE_SHA1);
-
- err = block_writer_add(&writer, &expected_refs[i]);
- cl_assert_equal_i(err, 0);
}
- err = block_writer_finish(&writer);
- cl_assert(err > 0);
+ cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_REF,
+ expected_refs, ARRAY_SIZE(expected_refs));
block_source_from_buf(&source, &data);
reftable_block_init(&block, &source, 0, 0, data.len,
@@ -453,6 +463,5 @@ void test_reftable_block__iterator(void)
reftable_ref_record_release(&ref);
reftable_iterator_destroy(&it);
reftable_block_release(&block);
- block_writer_release(&writer);
reftable_buf_release(&data);
}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 06/12] reftable/block: fix OOB write with bogus inflated log size
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
The "log" reftable block stores reflog information. This information is
compressed using zlib. The inflated size is stored in the block header
so that callers can easily learn ahead of time how large of a buffer
they have to allocate to inflate the data in a single pass. So to
reconstruct the full inflated block we:
- Copy over the header as-is, as it's not deflated.
- Append the inflated data to the buffer.
The inflated block size stored in the header also includes the length of
the header itself. So to figure out the bytes that should be inflated by
zlib we need to subtract the header size, which is trusted data, from
the block size, which is untrusted data derived from the block header.
While we do verify that we were able to inflate all data as expected, we
don't verify ahead of time that the encoded block length is larger than
the header length. This can lead to an underflow, which makes zlib
assume that it can write more data into the target buffer than we have
allocated. The result is an out-of-bounds write:
==1422297==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7c1ff6de5231 at pc 0x55555579a628 bp 0x7fffffff4f10 sp 0x7fffffff46d0
WRITE of size 4 at 0x7c1ff6de5231 thread T0
#0 0x55555579a627 in __asan_memcpy (./build/t/unit-tests+0x246627)
#1 0x55555598b093 in reftable_block_init ./build/../reftable/block.c:277:3
#2 0x555555813701 in test_reftable_block__corrupt_log_block_size ./build/../t/unit-tests/u-reftable-block.c:495:20
#3 0x5555557f684e in clar_run_test ./build/../t/unit-tests/clar/clar.c:335:3
#4 0x5555557f2e69 in clar_run_suite ./build/../t/unit-tests/clar/clar.c:431:3
#5 0x5555557f2882 in clar_test_run ./build/../t/unit-tests/clar/clar.c:636:4
#6 0x5555557f375f in clar_test ./build/../t/unit-tests/clar/clar.c:687:11
#7 0x5555557fa49d in cmd_main ./build/../t/unit-tests/unit-test.c:62:8
#8 0x55555584af4a in main ./build/../common-main.c:9:11
#9 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#10 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#11 0x555555694c24 in _start (./build/t/unit-tests+0x140c24)
0x7c1ff6de5231 is located 0 bytes after 1-byte region [0x7c1ff6de5230,0x7c1ff6de5231)
allocated by thread T0 here:
#0 0x55555579db1b in realloc.part.0 asan_malloc_linux.cpp.o
#1 0x5555559868d7 in reftable_realloc ./build/../reftable/basics.c:36:9
#2 0x55555598a98f in reftable_alloc_grow ./build/../reftable/basics.h:229:10
#3 0x55555598ae58 in reftable_block_init ./build/../reftable/block.c:269:3
#4 0x555555813701 in test_reftable_block__corrupt_log_block_size ./build/../t/unit-tests/u-reftable-block.c:495:20
#5 0x5555557f684e in clar_run_test ./build/../t/unit-tests/clar/clar.c:335:3
#6 0x5555557f2e69 in clar_run_suite ./build/../t/unit-tests/clar/clar.c:431:3
#7 0x5555557f2882 in clar_test_run ./build/../t/unit-tests/clar/clar.c:636:4
#8 0x5555557f375f in clar_test ./build/../t/unit-tests/clar/clar.c:687:11
#9 0x5555557fa49d in cmd_main ./build/../t/unit-tests/unit-test.c:62:8
#10 0x55555584af4a in main ./build/../common-main.c:9:11
#11 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#12 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#13 0x555555694c24 in _start (./build/t/unit-tests+0x140c24)
SUMMARY: AddressSanitizer: heap-buffer-overflow (./build/t/unit-tests+0x246627) in __asan_memcpy
Shadow bytes around the buggy address:
0x7c1ff6de4f80: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7c1ff6de5000: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7c1ff6de5080: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7c1ff6de5100: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7c1ff6de5180: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fd
=>0x7c1ff6de5200: fa fa 04 fa fa fa[01]fa fa fa fa fa fa fa fa fa
0x7c1ff6de5280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c1ff6de5300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c1ff6de5380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c1ff6de5400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c1ff6de5480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Fix the bug by adding a sanity check and add a unit test.
Reported-by: oxsignal <awo@kakao.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/block.c | 9 +++++++++
t/unit-tests/u-reftable-block.c | 32 ++++++++++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/reftable/block.c b/reftable/block.c
index 920b3f4486..b86cb9ec5a 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -260,6 +260,15 @@ int reftable_block_init(struct reftable_block *block,
goto done;
}
+ /*
+ * Verify that the block size covers at least the table header, block
+ * header and the 2 byte restart counter.
+ */
+ if (block_size < header_size + 4 + 2) {
+ err = REFTABLE_FORMAT_ERROR;
+ goto done;
+ }
+
if (block_type == REFTABLE_BLOCK_TYPE_LOG) {
uint32_t block_header_skip = 4 + header_size;
uLong dst_len = block_size - block_header_skip;
diff --git a/t/unit-tests/u-reftable-block.c b/t/unit-tests/u-reftable-block.c
index f4e926ce3a..088162483e 100644
--- a/t/unit-tests/u-reftable-block.c
+++ b/t/unit-tests/u-reftable-block.c
@@ -465,3 +465,35 @@ void test_reftable_block__iterator(void)
reftable_block_release(&block);
reftable_buf_release(&data);
}
+
+void test_reftable_block__corrupt_log_block_size(void)
+{
+ struct reftable_block_source source = { 0 };
+ struct reftable_record rec = {
+ .type = REFTABLE_BLOCK_TYPE_LOG,
+ .u.log = {
+ .refname = (char *) "refs/heads/main",
+ .update_index = 1,
+ .value_type = REFTABLE_LOG_UPDATE,
+ },
+ };
+ struct reftable_block block = { 0 };
+ struct reftable_buf data = REFTABLE_BUF_INIT;
+
+ cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_LOG, &rec, 1);
+
+ /*
+ * Log blocks store their inflated size as a big-endian 24-bit integer
+ * right after the one-byte block type. Rewrite it to claim a size that
+ * is smaller than the block header.
+ */
+ reftable_put_be24((uint8_t *) data.buf + 1, 1);
+
+ block_source_from_buf(&source, &data);
+ cl_assert_equal_i(reftable_block_init(&block, &source, 0, 0, data.len,
+ REFTABLE_HASH_SIZE_SHA1, REFTABLE_BLOCK_TYPE_LOG),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_block_release(&block);
+ reftable_buf_release(&data);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 07/12] reftable/block: fix OOB read with bogus block size
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
The block size is read from the block header, which is untrusted data.
We use it without verification to access the restart count at the end of
the block as well as to compute the restart table offset. With a bogus
block size that exceeds the data we have actually read this can lead to
an out-of-bounds read:
==2274138==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7c3ff6de2e3f at pc 0x55555598c6ea bp 0x7fffffff4ee0 sp 0x7fffffff4ed8
READ of size 1 at 0x7c3ff6de2e3f thread T0
#0 0x55555598c6e9 in reftable_get_be16 /home/pks/Development/git/build/../reftable/basics.h:119:20
#1 0x55555598c252 in reftable_block_init /home/pks/Development/git/build/../reftable/block.c:343:18
#2 0x555555813c70 in test_reftable_block__corrupt_block_size /home/pks/Development/git/build/../t/unit-tests/u-reftable-block.c:531:20
#3 0x5555557f684e in clar_run_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:335:3
#4 0x5555557f2e69 in clar_run_suite /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:431:3
#5 0x5555557f2882 in clar_test_run /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:636:4
#6 0x5555557f375f in clar_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:687:11
#7 0x5555557fa49d in cmd_main /home/pks/Development/git/build/../t/unit-tests/unit-test.c:62:8
#8 0x55555584b8aa in main /home/pks/Development/git/build/../common-main.c:9:11
#9 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b284) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
#10 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b337) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
#11 0x555555694c24 in _start (/home/pks/Development/git/build/t/unit-tests+0x140c24)
0x7c3ff6de2e3f is located 0 bytes after 47-byte region [0x7c3ff6de2e10,0x7c3ff6de2e3f)
allocated by thread T0 here:
#0 0x55555579e95b in malloc (/home/pks/Development/git/build/t/unit-tests+0x24a95b)
#1 0x5555559871c2 in reftable_malloc /home/pks/Development/git/build/../reftable/basics.c:24:9
#2 0x5555559872e8 in reftable_calloc /home/pks/Development/git/build/../reftable/basics.c:54:6
#3 0x55555598f0d3 in reftable_buf_read_data /home/pks/Development/git/build/../reftable/blocksource.c:67:2
#4 0x55555598ea7e in block_source_read_data /home/pks/Development/git/build/../reftable/blocksource.c:41:19
#5 0x55555598c555 in read_block /home/pks/Development/git/build/../reftable/block.c:224:9
#6 0x55555598b69e in reftable_block_init /home/pks/Development/git/build/../reftable/block.c:258:9
#7 0x555555813c70 in test_reftable_block__corrupt_block_size /home/pks/Development/git/build/../t/unit-tests/u-reftable-block.c:531:20
#8 0x5555557f684e in clar_run_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:335:3
#9 0x5555557f2e69 in clar_run_suite /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:431:3
#10 0x5555557f2882 in clar_test_run /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:636:4
#11 0x5555557f375f in clar_test /home/pks/Development/git/build/../t/unit-tests/clar/clar.c:687:11
#12 0x5555557fa49d in cmd_main /home/pks/Development/git/build/../t/unit-tests/unit-test.c:62:8
#13 0x55555584b8aa in main /home/pks/Development/git/build/../common-main.c:9:11
#14 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b284) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
#15 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/8kvxvr3pmsypxiypq4g8zy13glnfr7nx-glibc-2.42-67/lib/libc.so.6+0x2b337) (BuildId: 5a702452a01df1d7d50ce0663acec7be3c71fd4d)
#16 0x555555694c24 in _start (/home/pks/Development/git/build/t/unit-tests+0x140c24)
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/pks/Development/git/build/../reftable/basics.h:119:20 in reftable_get_be16
Shadow bytes around the buggy address:
0x7c3ff6de2b80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x7c3ff6de2c00: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x7c3ff6de2c80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
0x7c3ff6de2d00: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
0x7c3ff6de2d80: fa fa 00 00 00 00 00 00 fa fa fd fd fd fd fd fd
=>0x7c3ff6de2e00: fa fa 00 00 00 00 00[07]fa fa fa fa fa fa fa fa
0x7c3ff6de2e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c3ff6de2f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c3ff6de2f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c3ff6de3000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7c3ff6de3080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Verify that the claimed block size fits into the block data before using
it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/block.c | 9 +++++++++
t/unit-tests/u-reftable-block.c | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/reftable/block.c b/reftable/block.c
index b86cb9ec5a..4d6b11c2e7 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -340,6 +340,15 @@ int reftable_block_init(struct reftable_block *block,
full_block_size = block_size;
}
+ /*
+ * Ensure that we have sufficient data available now to satisfy the
+ * claimed block size.
+ */
+ if (block_size > block->block_data.len) {
+ err = REFTABLE_FORMAT_ERROR;
+ goto done;
+ }
+
restart_count = reftable_get_be16(block->block_data.data + block_size - 2);
restart_off = block_size - 2 - 3 * restart_count;
diff --git a/t/unit-tests/u-reftable-block.c b/t/unit-tests/u-reftable-block.c
index 088162483e..6995eb8bf1 100644
--- a/t/unit-tests/u-reftable-block.c
+++ b/t/unit-tests/u-reftable-block.c
@@ -497,3 +497,41 @@ void test_reftable_block__corrupt_log_block_size(void)
reftable_block_release(&block);
reftable_buf_release(&data);
}
+
+void test_reftable_block__corrupt_block_size(void)
+{
+ struct reftable_block_source source = { 0 };
+ struct reftable_record rec = {
+ .type = REFTABLE_BLOCK_TYPE_REF,
+ .u.ref = {
+ .value_type = REFTABLE_REF_VAL1,
+ .refname = (char *) "refs/heads/main",
+ },
+ };
+ struct reftable_block block = { 0 };
+ struct reftable_buf data = REFTABLE_BUF_INIT;
+ uint32_t block_size;
+ unsigned char *p;
+
+ cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_REF, &rec, 1);
+
+ /*
+ * The block size is stored as a big-endian 24-bit integer right after
+ * the one-byte block type at the start of the block. Corrupt it to
+ * claim a size that is larger than the data we actually have. Reading
+ * the restart count and restart table relative to such a bogus block
+ * size must not access out-of-bounds memory.
+ */
+ p = (unsigned char *) data.buf + 1;
+ block_size = reftable_get_be24(p);
+ cl_assert_equal_i(block_size, 47);
+ reftable_put_be24(p, block_size + 1);
+
+ block_source_from_buf(&source, &data);
+ cl_assert_equal_i(reftable_block_init(&block, &source, 0, 0, data.len,
+ REFTABLE_HASH_SIZE_SHA1, REFTABLE_BLOCK_TYPE_REF),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_block_release(&block);
+ reftable_buf_release(&data);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 08/12] reftable/block: fix OOB read with bogus restart count
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
The restart count is stored in the last two bytes of a block. We use it
without verification to compute the offset of the restart table. With a
bogus restart count that is large enough this computation underflows,
and the subsequent reads via the restart table access out-of-bounds
memory:
==129439==ERROR: AddressSanitizer: SEGV on unknown address 0x7d90f6dcd0ad (pc 0x55555598ce89 bp 0x7fffffff4ed0 sp 0x7fffffff4e80 T0)
==129439==The signal is caused by a READ memory access.
#0 0x55555598ce89 in reftable_get_be24 ./git/build/../reftable/basics.h:125:9
#1 0x55555598eabf in block_restart_offset ./git/build/../reftable/block.c:407:9
#2 0x55555598e5d5 in restart_needle_less ./git/build/../reftable/block.c:431:17
#3 0x5555559887e2 in binsearch ./git/build/../reftable/basics.c:165:13
#4 0x55555598dfec in block_iter_seek_key ./git/build/../reftable/block.c:529:6
#5 0x555555814517 in test_reftable_block__corrupt_restart_count ./git/build/../t/unit-tests/u-reftable-block.c:593:15
#6 0x5555557f684e in clar_run_test ./git/build/../t/unit-tests/clar/clar.c:335:3
#7 0x5555557f2e69 in clar_run_suite ./git/build/../t/unit-tests/clar/clar.c:431:3
#8 0x5555557f2882 in clar_test_run ./git/build/../t/unit-tests/clar/clar.c:636:4
#9 0x5555557f375f in clar_test ./git/build/../t/unit-tests/clar/clar.c:687:11
#10 0x5555557fa49d in cmd_main ./git/build/../t/unit-tests/unit-test.c:62:8
#11 0x55555584c12a in main ./git/build/../common-main.c:9:11
#12 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#13 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#14 0x555555694c24 in _start (./git/build/t/unit-tests+0x140c24)
==129439==Register values:
rax = 0x00007d90f6dcd0ad rbx = 0x00007fffffff4f20 rcx = 0xf2f2f2f8f2f2f2f8 rdx = 0x0000000000000000
rdi = 0x00007d90f6dcd0ad rsi = 0x0000000000007fff rbp = 0x00007fffffff4ed0 rsp = 0x00007fffffff4e80
r8 = 0x0000000000000000 r9 = 0x0000000000000000 r10 = 0x0000000000000000 r11 = 0x0000000000000017
r12 = 0x00007fffffff58e8 r13 = 0x0000000000000001 r14 = 0x00007ffff7ffd000 r15 = 0x00005555560550b0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ./git/build/../reftable/basics.h:125:9 in reftable_get_be24
Verify that the restart table actually fits into the block.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/block.c | 4 ++++
t/unit-tests/u-reftable-block.c | 33 +++++++++++++++++++++++++++++++++
2 files changed, 37 insertions(+)
diff --git a/reftable/block.c b/reftable/block.c
index 4d6b11c2e7..4d285aefd7 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -351,6 +351,10 @@ int reftable_block_init(struct reftable_block *block,
restart_count = reftable_get_be16(block->block_data.data + block_size - 2);
restart_off = block_size - 2 - 3 * restart_count;
+ if (restart_off < header_size + 4 || restart_off > block_size - 2) {
+ err = REFTABLE_FORMAT_ERROR;
+ goto done;
+ }
block->block_type = block_type;
block->hash_size = hash_size;
diff --git a/t/unit-tests/u-reftable-block.c b/t/unit-tests/u-reftable-block.c
index 6995eb8bf1..d77048cdd0 100644
--- a/t/unit-tests/u-reftable-block.c
+++ b/t/unit-tests/u-reftable-block.c
@@ -535,3 +535,36 @@ void test_reftable_block__corrupt_block_size(void)
reftable_block_release(&block);
reftable_buf_release(&data);
}
+
+void test_reftable_block__corrupt_restart_count(void)
+{
+ struct reftable_block_source source = { 0 };
+ struct reftable_record rec = {
+ .type = REFTABLE_BLOCK_TYPE_REF,
+ .u.ref = {
+ .value_type = REFTABLE_REF_VAL1,
+ .refname = (char *) "refs/heads/main",
+ },
+ };
+ struct reftable_block block = { 0 };
+ struct reftable_buf data = REFTABLE_BUF_INIT;
+ int block_size;
+
+ block_size = cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_REF, &rec, 1);
+
+ /*
+ * Corrupt the restart count to claim a bogus number of restart points.
+ * Note that this would only cause us to perform an out-of-bounds
+ * access when seeking into the block, but we want to refuse such a
+ * block outright.
+ */
+ reftable_put_be16((uint8_t *) data.buf + block_size - 2, 0xffff);
+
+ block_source_from_buf(&source, &data);
+ cl_assert_equal_i(reftable_block_init(&block, &source, 0, 0, data.len,
+ REFTABLE_HASH_SIZE_SHA1, REFTABLE_BLOCK_TYPE_REF),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_block_release(&block);
+ reftable_buf_release(&data);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 09/12] reftable/block: fix use of uninitialized memory when binsearch fails
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
When doing the binary search through our restart offsets we may hit an
error in case `restart_needle_less()` fails to decode the record at the
given offset. While we correctly detect this case and error out, it will
cause us to call `reftable_record_release()` on the yet-uninitialized
record.
Fix this by initializing the record earlier.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/block.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/reftable/block.c b/reftable/block.c
index 4d285aefd7..89efce8751 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -517,6 +517,10 @@ int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)
int err = 0;
size_t i;
+ err = reftable_record_init(&rec, reftable_block_type(it->block));
+ if (err < 0)
+ goto done;
+
/*
* Perform a binary search over the block's restart points, which
* avoids doing a linear scan over the whole block. Like this, we
@@ -558,10 +562,6 @@ int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)
else
it->next_off = it->block->header_off + 4;
- err = reftable_record_init(&rec, reftable_block_type(it->block));
- if (err < 0)
- goto done;
-
/*
* We're looking for the last entry less than the wanted key so that
* the next call to `block_reader_next()` would yield the wanted
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 10/12] reftable/block: fix OOB read with bogus restart offset
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
Restart points encode records in a given block that do not use prefix
compression and that can thus immediately be seeked to. These offsets
are encoded in the restart table, where each offset needs to point at
one of the records of the block. We do not verify this though, so a
bogus restart offset may cause an out-of-bounds read:
==1472280==ERROR: AddressSanitizer: SEGV on unknown address 0x7d8ff7de5f7f (pc 0x55555599502b bp 0x7fffffff4df0 sp 0x7fffffff4d40 T0)
==1472280==The signal is caused by a READ memory access.
#0 0x55555599502b in get_var_int ./git/build/../reftable/record.c:30:6
#1 0x555555995c2a in reftable_decode_keylen ./git/build/../reftable/record.c:177:6
#2 0x55555598e85c in restart_needle_less ./git/build/../reftable/block.c:455:6
#3 0x55555598895f in binsearch ./git/build/../reftable/basics.c:175:9
#4 0x55555598e189 in block_iter_seek_key ./git/build/../reftable/block.c:543:6
#5 0x555555814aee in test_reftable_block__corrupt_restart_offset ./git/build/../t/unit-tests/u-reftable-block.c:636:20
#6 0x5555557f684e in clar_run_test ./git/build/../t/unit-tests/clar/clar.c:335:3
#7 0x5555557f2e69 in clar_run_suite ./git/build/../t/unit-tests/clar/clar.c:431:3
#8 0x5555557f2882 in clar_test_run ./git/build/../t/unit-tests/clar/clar.c:636:4
#9 0x5555557f375f in clar_test ./git/build/../t/unit-tests/clar/clar.c:687:11
#10 0x5555557fa49d in cmd_main ./git/build/../t/unit-tests/unit-test.c:62:8
#11 0x55555584c25a in main ./git/build/../common-main.c:9:11
#12 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#13 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#14 0x555555694c24 in _start (./git/build/t/unit-tests+0x140c24)
==1472280==Register values:
rax = 0x00007d8ff7de5f7f rbx = 0x00007fffffff4e00 rcx = 0x00007d8ff7de5f80 rdx = 0x00007bfff5b6af60
rdi = 0x00007bfff5b6af40 rsi = 0x00007bfff592dfa0 rbp = 0x00007fffffff4df0 rsp = 0x00007fffffff4d40
r8 = 0x00000000ff00002b r9 = 0x00007d8ff7de5f7f r10 = 0x00000f7ffeb25bf0 r11 = 0xf3f30000f1f1f1f1
r12 = 0x00007fffffff58f8 r13 = 0x0000000000000001 r14 = 0x00007ffff7ffd000 r15 = 0x0000555556055fd0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ./git/build/../reftable/record.c:30:6 in get_var_int
Guard against such restart offsets and signal an error to the caller via
`args.error`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/block.c | 9 +++++++++
t/unit-tests/u-reftable-block.c | 39 +++++++++++++++++++++++++++++++++++++++
2 files changed, 48 insertions(+)
diff --git a/reftable/block.c b/reftable/block.c
index 89efce8751..1fa81405d2 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -440,6 +440,15 @@ static int restart_needle_less(size_t idx, void *_args)
uint8_t extra;
int n;
+ /*
+ * The restart offset must point to a record, which is stored before
+ * the restart table. Verify that this is the case.
+ */
+ if (off >= args->block->restart_off) {
+ args->error = 1;
+ return -1;
+ }
+
/*
* Records at restart points are stored without prefix compression, so
* there is no need to fully decode the record key here. This removes
diff --git a/t/unit-tests/u-reftable-block.c b/t/unit-tests/u-reftable-block.c
index d77048cdd0..99b155d4bf 100644
--- a/t/unit-tests/u-reftable-block.c
+++ b/t/unit-tests/u-reftable-block.c
@@ -568,3 +568,42 @@ void test_reftable_block__corrupt_restart_count(void)
reftable_block_release(&block);
reftable_buf_release(&data);
}
+
+void test_reftable_block__corrupt_restart_offset(void)
+{
+ struct reftable_block_source source = { 0 };
+ struct reftable_record rec = {
+ .type = REFTABLE_BLOCK_TYPE_REF,
+ .u.ref = {
+ .value_type = REFTABLE_REF_VAL1,
+ .refname = (char *) "refs/heads/main",
+ },
+ };
+ struct reftable_block block = { 0 };
+ struct block_iter it = BLOCK_ITER_INIT;
+ struct reftable_buf want = REFTABLE_BUF_INIT;
+ struct reftable_buf data = REFTABLE_BUF_INIT;
+
+ cl_reftable_write_block(&data, REFTABLE_BLOCK_TYPE_REF, &rec, 1);
+
+ block_source_from_buf(&source, &data);
+ cl_must_pass(reftable_block_init(&block, &source, 0, 0, data.len,
+ REFTABLE_HASH_SIZE_SHA1, REFTABLE_BLOCK_TYPE_REF));
+
+ /*
+ * Corrupt the first restart offset, stored as a big-endian 24-bit
+ * integer at the start of the restart table, to point past the end of
+ * the records section. Seeking such a block must fail gracefully.
+ */
+ reftable_put_be24((uint8_t *) block.block_data.data + block.restart_off,
+ 0xffffff);
+
+ block_iter_init(&it, &block);
+ cl_must_pass(reftable_buf_addstr(&want, "refs/heads/main"));
+ cl_assert_equal_i(block_iter_seek_key(&it, &want), REFTABLE_FORMAT_ERROR);
+
+ reftable_buf_release(&want);
+ block_iter_close(&it);
+ reftable_block_release(&block);
+ reftable_buf_release(&data);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 11/12] reftable/table: fix NULL pointer access when seeking to bogus offsets
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
When seeking an iterator to an arbitrary offset we may return a positive
value in case the offset points beyond the block. This makes sense when
iterating through multiple blocks of the same section, as the positive
value indicates to us that we're at the end of the table.
But when the offset originates from a section or index offset it is
supposed to point at a valid block, so an out-of-bounds value means that
the table is corrupt. Treating it as a normal end-of-iteration causes us
to silently report an empty section instead of surfacing the corruption,
and we are left with a partially-initialized block. This may later on
cause a NULL pointer exception:
==1486841==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55555598e02c bp 0x7fffffff4eb0 sp 0x7fffffff4e70 T0)
==1486841==The signal is caused by a READ memory access.
==1486841==Hint: address points to the zero page.
#0 0x55555598e02c in reftable_block_type ./git/build/../reftable/block.c:392:9
#1 0x55555598ee6e in block_iter_seek_key ./git/build/../reftable/block.c:536:35
#2 0x5555559ae553 in table_iter_seek_linear ./git/build/../reftable/table.c:344:8
#3 0x5555559adbff in table_iter_seek ./git/build/../reftable/table.c:450:9
#4 0x5555559ada9c in table_iter_seek_void ./git/build/../reftable/table.c:460:9
#5 0x555555992872 in reftable_iterator_seek_log_at ./git/build/../reftable/iter.c:281:9
#6 0x555555992953 in reftable_iterator_seek_log ./git/build/../reftable/iter.c:287:9
#7 0x55555583aa78 in test_reftable_table__seek_invalid_log_offset ./git/build/../t/unit-tests/u-reftable-table.c:257:20
#8 0x5555557f684e in clar_run_test ./git/build/../t/unit-tests/clar/clar.c:335:3
#9 0x5555557f2e69 in clar_run_suite ./git/build/../t/unit-tests/clar/clar.c:431:3
#10 0x5555557f2882 in clar_test_run ./git/build/../t/unit-tests/clar/clar.c:636:4
#11 0x5555557f375f in clar_test ./git/build/../t/unit-tests/clar/clar.c:687:11
#12 0x5555557fa49d in cmd_main ./git/build/../t/unit-tests/unit-test.c:62:8
#13 0x55555584cffa in main ./git/build/../common-main.c:9:11
#14 0x7ffff7a2b284 in __libc_start_call_main (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b284) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#15 0x7ffff7a2b337 in __libc_start_main@GLIBC_2.2.5 (/nix/store/57iz36553175g3178pvxjij8z5rcsd4n-glibc-2.42-61/lib/libc.so.6+0x2b337) (BuildId: 8ae0b698f2d4e727f569f64bb166e08ae30bd077)
#16 0x555555694c24 in _start (./git/build/t/unit-tests+0x140c24)
==1486841==Register values:
rax = 0x0000000000000000 rbx = 0x00007fffffff4ec0 rcx = 0x0000000000000000 rdx = 0x00007cfff6e2bd58
rdi = 0x00007cfff6e2bd58 rsi = 0x00007bfff5da1020 rbp = 0x00007fffffff4eb0 rsp = 0x00007fffffff4e70
r8 = 0x0000000000000000 r9 = 0x0000000000000002 r10 = 0x0000000000000000 r11 = 0x0000000000000017
r12 = 0x00007fffffff5908 r13 = 0x0000000000000001 r14 = 0x00007ffff7ffd000 r15 = 0x0000555556056e90
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ./git/build/../reftable/block.c:392:9 in reftable_block_type
==1486841==ABORTING
Fix this by returning a proper error in `table_iter_seek_to()` when the
offset ranges beyond the block.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/table.c | 2 ++
t/unit-tests/u-reftable-table.c | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 65 insertions(+)
diff --git a/reftable/table.c b/reftable/table.c
index 56362df0ed..f4bc86a29d 100644
--- a/reftable/table.c
+++ b/reftable/table.c
@@ -242,6 +242,8 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)
int err;
err = table_init_block(ti->table, &ti->block, off, typ);
+ if (err > 0)
+ return REFTABLE_FORMAT_ERROR;
if (err != 0)
return err;
diff --git a/t/unit-tests/u-reftable-table.c b/t/unit-tests/u-reftable-table.c
index 14fae8b199..c7dca45e70 100644
--- a/t/unit-tests/u-reftable-table.c
+++ b/t/unit-tests/u-reftable-table.c
@@ -1,8 +1,11 @@
#include "unit-test.h"
#include "lib-reftable.h"
+#include "reftable/basics.h"
+#include "reftable/block.h"
#include "reftable/blocksource.h"
#include "reftable/constants.h"
#include "reftable/iter.h"
+#include "reftable/reftable-error.h"
#include "reftable/table.h"
#include "strbuf.h"
@@ -199,3 +202,63 @@ void test_reftable_table__block_iterator(void)
reftable_buf_release(&buf);
reftable_free(records);
}
+
+void test_reftable_table__seek_invalid_log_offset(void)
+{
+ struct reftable_ref_record refs[] = {
+ {
+ .refname = (char *) "refs/heads/main",
+ .value_type = REFTABLE_REF_VAL1,
+ .value.val1 = { 42 },
+ },
+ };
+ struct reftable_log_record logs[] = {
+ {
+ .refname = (char *) "refs/heads/main",
+ .update_index = 1,
+ .value_type = REFTABLE_LOG_UPDATE,
+ .value.update = {
+ .name = (char *) "user",
+ .email = (char *) "user@example.com",
+ .message = (char *) "message\n",
+ },
+ },
+ };
+ struct reftable_block_source source = { 0 };
+ struct reftable_log_record log = { 0 };
+ struct reftable_iterator it = { 0 };
+ struct reftable_table *table;
+ struct reftable_buf buf = REFTABLE_BUF_INIT;
+ size_t fsize = footer_size(1);
+ uint8_t *footer;
+
+ cl_reftable_write_to_buf(&buf, refs, ARRAY_SIZE(refs),
+ logs, ARRAY_SIZE(logs), NULL);
+
+ /*
+ * Corrupt the log section offset stored in the footer so that it points
+ * past the end of the table. The footer is checksummed, so we also have
+ * to recompute and rewrite the CRC.
+ */
+ footer = (uint8_t *) buf.buf + buf.len - fsize;
+ reftable_put_be64(footer + header_size(1) + 24, UINT64_MAX);
+ reftable_put_be32(footer + fsize - 4, crc32(0, footer, fsize - 4));
+
+ block_source_from_buf(&source, &buf);
+ cl_must_pass(reftable_table_new(&table, &source, "name"));
+
+ /*
+ * Seeking the log iterator must not crash even though the log section
+ * offset is bogus. As the offset points past the end of the table we
+ * know that the table is corrupt, so the seek must report a format
+ * error instead of pretending that the section is empty.
+ */
+ reftable_table_init_log_iterator(table, &it);
+ cl_assert_equal_i(reftable_iterator_seek_log(&it, ""),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_log_record_release(&log);
+ reftable_iterator_destroy(&it);
+ reftable_table_decref(table);
+ reftable_buf_release(&buf);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* [PATCH v3 12/12] reftable/table: fix OOB read on truncated table
From: Patrick Steinhardt @ 2026-07-03 12:58 UTC (permalink / raw)
To: git; +Cc: oxsignal, Toon Claes, Christian Couder
In-Reply-To: <20260703-pks-reftable-hardening-v3-0-b87c555b9920@pks.im>
When opening a table we compute the size of its data section by
subtracting the footer size from the file size. We do not verify that
the file is actually large enough to contain both the header and the
footer though. With a truncated table the subtraction can thus
underflow, causing us to read the footer out of bounds:
SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/pks/Development/git/build/t/unit-tests+0x2479a4) in __asan_memcpy
Shadow bytes around the buggy address:
0x7ccff6e0de80: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x7ccff6e0df00: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
0x7ccff6e0df80: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x7ccff6e0e000: fd fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd
0x7ccff6e0e080: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa
=>0x7ccff6e0e100: fa fa fa fa fa[fa]00 00 00 00 00 00 00 00 00 00
0x7ccff6e0e180: 00 00 00 00 00 00 00 04 fa fa fa fa fa fa fa fa
0x7ccff6e0e200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7ccff6e0e280: 00 00 fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ccff6e0e300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ccff6e0e380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==1500371==ABORTING
Verify that the file is large enough to contain both the header and the
footer before computing the table size.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
reftable/table.c | 5 +++++
t/unit-tests/u-reftable-table.c | 28 ++++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/reftable/table.c b/reftable/table.c
index f4bc86a29d..b4d3f9e211 100644
--- a/reftable/table.c
+++ b/reftable/table.c
@@ -562,6 +562,11 @@ int reftable_table_new(struct reftable_table **out,
goto done;
}
+ if (file_size < header_size(t->version) + footer_size(t->version)) {
+ err = REFTABLE_FORMAT_ERROR;
+ goto done;
+ }
+
t->size = file_size - footer_size(t->version);
t->source = *source;
t->name = reftable_strdup(name);
diff --git a/t/unit-tests/u-reftable-table.c b/t/unit-tests/u-reftable-table.c
index c7dca45e70..28b0ef5258 100644
--- a/t/unit-tests/u-reftable-table.c
+++ b/t/unit-tests/u-reftable-table.c
@@ -262,3 +262,31 @@ void test_reftable_table__seek_invalid_log_offset(void)
reftable_table_decref(table);
reftable_buf_release(&buf);
}
+
+void test_reftable_table__new_with_truncated_table(void)
+{
+ struct reftable_ref_record refs[] = {
+ {
+ .refname = (char *) "refs/heads/main",
+ .value_type = REFTABLE_REF_VAL1,
+ .value.val1 = { 42 },
+ },
+ };
+ struct reftable_block_source source = { 0 };
+ struct reftable_table *table;
+ struct reftable_buf buf = REFTABLE_BUF_INIT;
+
+ cl_reftable_write_to_buf(&buf, refs, ARRAY_SIZE(refs), NULL, 0, NULL);
+
+ /*
+ * Truncate the table so that it is large enough to read the header, but
+ * too small to also contain the footer.
+ */
+ buf.len = footer_size(1) - 1;
+ block_source_from_buf(&source, &buf);
+
+ cl_assert_equal_i(reftable_table_new(&table, &source, "name"),
+ REFTABLE_FORMAT_ERROR);
+
+ reftable_buf_release(&buf);
+}
--
2.55.0.795.g602f6c329a.dirty
^ permalink raw reply related
* Re: Unexpected recursion in 'git rm'
From: Matt Hunter @ 2026-07-03 13:04 UTC (permalink / raw)
To: Евгений Плискин,
Patrick Steinhardt
Cc: git
In-Reply-To: <1978773121.20260703123754@gmail.com>
On Fri Jul 3, 2026 at 5:37 AM EDT, Евгений Плискин wrote:
>> Hi - I threw a quick test repo together, but did not see the result you describe. Could you produce a script or series of commands to reproduce the problem?
> Hi. Thank you for your reply.
> I was advised meantime that indeed this behaviour is expected:
>
> This is expected behaviour, as the argument to git-rm(1) is a pathspec,
> and "*" matches directory separators by default, see also gitglossary(7)
> under "pathspec":
Yup, I see that now, and that was the difference maker in my test. I
had json files in the directory I was working, as well as at nested
paths. My (non-windows) shell globbed to pass just the upper files.
To address Patrick's question in <akdzSHrJ4DfdUWoS@pks.im>:
> Could you maybe clarify which part of git-rm(1) made you think that this
> wouldn't happen?
I'll say that I _don't_ think the man page is unclear in this regard
(having given it a look just now). To me, this is just one of the
aspects of git that is integrated with its unix counterpart well enough
(git-rm vs. rm) that you can forget the finer details like this fairly
easily.
And since the command is so straightforward, I couldn't tell you the
last time I pulled up git-rm(1) as a reference.
^ permalink raw reply
* Re: Programmatically edit the git rebase sequence?
From: brian m. carlson @ 2026-07-03 13:42 UTC (permalink / raw)
To: Matthias Beyer; +Cc: git, neikos
In-Reply-To: <akei64goQf3nFhX4@hikari>
[-- Attachment #1: Type: text/plain, Size: 2243 bytes --]
On 2026-07-03 at 12:02:33, Matthias Beyer wrote:
> Hi git people,
>
> in a recent conversation at work, the question of how to
> programmatically edit the git-rebase sequence came up.
>
> Example use case:
>
> I have a branch that touches a number of files, adds some files and
> removes some files.
> When rebasing, I want to split all commits that touched a certain subset
> of files, for the clearity of the history.
>
> I look at the output of
>
> git log master..mybranch --oneline --diff-filter=M -- "./subdir/*.rs"
>
> to find all commits in that subdir that only touched the files. All of
> these commits are to be "edit"ed.
>
> Now I fire up `git rebase -i master` and manually(!) match the list from
> above `git-log` call and find the respective commits to edit them.
>
> Is there a way I am not aware of to do that manual step programatically?
> Something like
>
> git rebase -i master --edit-commits="$(git log master..mybranch --diff-filter=M --format="%H" -- "./subdir/*.rs")"
>
> would be convenient here, although I would understand if that is too
> much clutter for the already very heavy git CLI interface :-)
Yes, such a thing exists. You want `GIT_SEQUENCE_EDITOR`, which is an
`EDITOR`-like command that edits the rebase list in place. So tools
like `ed`, `ex`, `sed -i`, `perl -i`, or `ruby -i` would be useful here.
So you might want something like this (untested):
GIT_SEQUENCE_EDITOR="perl -pi -e 's/^pick ($(git log master..mybranch --diff-filter=M --format="%h" -- "./subdir/*.rs" | paste -d '\''|'\'' -s -))/edit \$1/'" \
git rebase -i master
Note the use of `%h`, since by default the object IDs are abbreviated.
If you want something simpler, you can also write a shell script which
edits the first argument in place and specify that. Arbitrary shell is
allowed in `GIT_SEQUENCE_EDITOR`, much like in `EDITOR` and `VISUAL`.
I personally use this alias, which explicitly does not edit the sequence
list, to automatically squash in all squash and fixup commits without
prompting:
srebase = "!f() { GIT_SEQUENCE_EDITOR=true git rebase -m -i --autosquash \"$@\"; };f"
--
brian m. carlson (they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 325 bytes --]
^ permalink raw reply
* Re: [PATCH 7/9] http: discard hash in dumb-http http_object_request
From: brian m. carlson @ 2026-07-03 13:47 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Jeff King, git
In-Reply-To: <akecqPq4F702E8Cq@pks.im>
[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]
On 2026-07-03 at 11:27:36, Patrick Steinhardt wrote:
> On Thu, Jul 02, 2026 at 04:07:07AM -0400, Jeff King wrote:
> > The flag handling could be removed if the hash-discard function were
> > idempotent. This could be done easily-ish by having the underlying
> > hash functions (like the ones in sha256/openssl.h) set the context
> > pointer to NULL after free-ing. But it's something that every platform
> > implementation would have to remember to do, and the benefit for the
> > callers is not that huge (it would let us shave a few lines here and
> > probably in a few other spots).
>
> This answers an earlier question of mine. It would indeed be great if it
> was idempotent -- I've been bitten by interfaces like this once too
> much, where you have to be very careful to manage the lifetime of a
> specific object. The prime example of this are (were? I don't quite
> recall whether we fixed that interface) reference transactions, and that
> caused a bunch of bugs in the past.
Yes, that would be fantastic. The Rust code will need a few fixes as
well (which I will send on top of this one when it's picked up) and it
really simplifies our Drop implementation if I can just do
`git_hash_discard`. Otherwise, I need to keep track of whether we've
already called one of the final functions or not to avoid a double free.
--
brian m. carlson (they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 325 bytes --]
^ permalink raw reply
* Re: [PATCH v3 4/5] builtin/refs: add "create" subcommand
From: Toon Claes @ 2026-07-03 14:19 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Junio C Hamano
In-Reply-To: <20260630-pks-refs-writing-subcommands-v3-4-deb04de1ecef@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
> The "update" subcommand cannot only update an existing reference, but it
> can also create new branches and delete existing branches by specifying
> the all-zeroes object ID as either old or new value. Despite that, we
> already have the "delete" subcommand as a handy shortcut so that a user
> can easily delete a branch. This relieves them of needing to understand
> the more arcane uses of the "update" command, and of counting the number
> of zeroes they need to pass.
>
> But while we have a "delete" subcommand, we don't have an equivalent
> that would allow the user to create a new branch, which creates a
> certain asymmetry.
>
> Add a new "create" subcommand to plug this gap.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> Documentation/git-refs.adoc | 5 ++
> builtin/refs.c | 52 +++++++++++++++
> t/meson.build | 1 +
> t/t1466-refs-create.sh | 151 ++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 209 insertions(+)
>
> diff --git a/Documentation/git-refs.adoc b/Documentation/git-refs.adoc
> index 6475bdcc62..e6a3528349 100644
> --- a/Documentation/git-refs.adoc
> +++ b/Documentation/git-refs.adoc
> @@ -20,6 +20,7 @@ git refs list [--count=<count>] [--shell|--perl|--python|--tcl]
> [ --stdin | (<pattern>...)]
> git refs exists <ref>
> git refs optimize [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>]
> +git refs create [--message=<reason>] [--no-deref] [--create-reflog] <ref> <new-value>
> git refs delete [--message=<reason>] [--no-deref] <ref> [<old-value>]
> git refs update [--message=<reason>] [--no-deref] [--create-reflog] <ref> <new-value> [<old-value>]
>
> @@ -53,6 +54,10 @@ optimize::
> usage. This subcommand is an alias for linkgit:git-pack-refs[1] and
> offers identical functionality.
>
> +create::
> + Create the given reference, which must not already exist, pointing at
> + `<new-value>`.
> +
> delete::
> Delete the given reference. This subcommand mirrors `git update-ref -d`
> (see linkgit:git-update-ref[1]). When `<old-value>` is given, the
> diff --git a/builtin/refs.c b/builtin/refs.c
> index 08453ae1c8..1ebaf30149 100644
> --- a/builtin/refs.c
> +++ b/builtin/refs.c
> @@ -21,6 +21,9 @@
> #define REFS_OPTIMIZE_USAGE \
> N_("git refs optimize " PACK_REFS_OPTS)
>
> +#define REFS_CREATE_USAGE \
> + N_("git refs create [--message=<reason>] [--no-deref] [--create-reflog] <ref> <new-value>")
> +
> #define REFS_DELETE_USAGE \
> N_("git refs delete [--message=<reason>] [--no-deref] <ref> [<old-value>]")
>
> @@ -181,6 +184,53 @@ static int cmd_refs_optimize(int argc, const char **argv, const char *prefix,
> return pack_refs_core(argc, argv, prefix, repo, refs_optimize_usage);
> }
>
> +static int cmd_refs_create(int argc, const char **argv, const char *prefix,
> + struct repository *repo)
> +{
> + static char const * const refs_create_usage[] = {
> + REFS_CREATE_USAGE,
> + NULL
> + };
> + const char *message = NULL;
> + unsigned flags = 0;
> + struct option opts[] = {
> + OPT_STRING(0, "message", &message, N_("reason"),
> + N_("reason of the update")),
> + OPT_BIT(0 ,"no-deref", &flags,
> + N_("update <refname> not the one it points to"),
> + REF_NO_DEREF),
Can `git refs create --no-deref` be used to create symrefs? Should we
add a test for that? Or can it not
I understand the symmetry, but does it make sense to ask the user to
create symrefs with `--no-deref`? Feels a bit obscure. The docs say:
`--no-deref`::
Operate on <ref> itself rather than the reference it points to via a
symbolic ref.
That's far from obvious for a user to realize they need to pass that
option if they want to create a symref.
> + OPT_BIT(0, "create-reflog", &flags, N_("create a reflog"),
> + REF_FORCE_CREATE_REFLOG),
> + OPT_END(),
> + };
> + struct object_id newoid;
> + const char *refname;
> + int ret;
> +
> + argc = parse_options(argc, argv, prefix, opts, refs_create_usage, 0);
> + if (argc != 2)
> + usage(_("create requires reference name and an object ID"));
> +
> + if (message && !*message)
> + die(_("refusing to perform update with empty message"));
> +
> + repo_config(repo, git_default_config, NULL);
> +
> + refname = argv[0];
> + if (repo_get_oid_with_flags(repo, argv[1], &newoid, GET_OID_SKIP_AMBIGUITY_CHECK))
> + die(_("invalid object ID: '%s'"), argv[1]);
> + if (is_null_oid(&newoid))
> + die(_("cannot create reference with null new object ID"));
> +
> + ret = refs_update_ref(get_main_ref_store(repo), message, refname,
> + &newoid, null_oid(repo->hash_algo), flags,
> + UPDATE_REFS_MSG_ON_ERR);
> +
> + if (ret < 0)
> + ret = 1;
> + return ret;
> +}
> +
> static int cmd_refs_delete(int argc, const char **argv, const char *prefix,
> struct repository *repo)
> {
> @@ -288,6 +338,7 @@ int cmd_refs(int argc,
> "git refs list " COMMON_USAGE_FOR_EACH_REF,
> REFS_EXISTS_USAGE,
> REFS_OPTIMIZE_USAGE,
> + REFS_CREATE_USAGE,
> REFS_DELETE_USAGE,
> REFS_UPDATE_USAGE,
> NULL,
> @@ -299,6 +350,7 @@ int cmd_refs(int argc,
> OPT_SUBCOMMAND("list", &fn, cmd_refs_list),
> OPT_SUBCOMMAND("exists", &fn, cmd_refs_exists),
> OPT_SUBCOMMAND("optimize", &fn, cmd_refs_optimize),
> + OPT_SUBCOMMAND("create", &fn, cmd_refs_create),
> OPT_SUBCOMMAND("delete", &fn, cmd_refs_delete),
> OPT_SUBCOMMAND("update", &fn, cmd_refs_update),
> OPT_END(),
> diff --git a/t/meson.build b/t/meson.build
> index 2063962dab..541e6f919c 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -225,6 +225,7 @@ integration_tests = [
> 't1463-refs-optimize.sh',
> 't1464-refs-delete.sh',
> 't1465-refs-update.sh',
> + 't1466-refs-create.sh',
> 't1500-rev-parse.sh',
> 't1501-work-tree.sh',
> 't1502-rev-parse-parseopt.sh',
> diff --git a/t/t1466-refs-create.sh b/t/t1466-refs-create.sh
> new file mode 100755
> index 0000000000..cfb21bf863
> --- /dev/null
> +++ b/t/t1466-refs-create.sh
> @@ -0,0 +1,151 @@
> +#!/bin/sh
> +
> +test_description='git refs create'
> +
> +. ./test-lib.sh
> +
> +setup_repo () {
> + git init "$1" &&
> + test_commit -C "$1" A &&
> + test_commit -C "$1" B
> +}
> +
> +test_ref_matches () {
> + git rev-parse "$1" >expect &&
> + echo "$2" >actual &&
> + test_cmp expect actual
> +}
> +
> +test_expect_success 'create a new reference' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git refs create refs/heads/foo $A &&
> + test_ref_matches refs/heads/foo "$A"
> + )
> +'
> +
> +test_expect_success 'create fails when the reference already exists' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + B=$(git rev-parse B) &&
> + git refs create refs/heads/foo $A &&
> + test_must_fail git refs create refs/heads/foo $B 2>err &&
> + test_grep "reference already exists" err &&
> + test_ref_matches refs/heads/foo "$A"
> + )
> +'
I was curious about this test:
test_expect_success 'create succeed when the reference exists with the same value' '
test_when_finished "rm -rf repo" &&
setup_repo repo &&
(
cd repo &&
A=$(git rev-parse A) &&
git refs create refs/heads/foo $A &&
git refs create refs/heads/foo $A &&
test_ref_matches refs/heads/foo "$A"
)
'
That fails. It that intentional?
> +
> +test_expect_success 'create with null new value fails' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + test_must_fail git refs create refs/heads/foo $ZERO_OID 2>err &&
> + test_grep "null new object ID" err &&
> + test_must_fail git refs exists refs/heads/foo
> + )
> +'
> +
> +test_expect_success 'create with invalid new value fails' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + test_must_fail git refs create refs/heads/foo invalid-oid 2>err &&
> + test_grep "invalid object ID" err &&
> + test_must_fail git refs exists refs/heads/foo
> + )
> +'
> +
> +test_expect_success 'create does not create a reflog by default' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git refs create refs/foo $A &&
> + test_must_fail git reflog exists refs/foo
> + )
> +'
> +
> +test_expect_success 'create creates a reflog with --create-reflog' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git refs create --create-reflog refs/foo $A &&
> + git reflog exists refs/foo
> + )
> +'
> +
> +test_expect_success 'create with message records reason in reflog' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git refs create --message="create reason" refs/heads/foo $A &&
> + git reflog show refs/heads/foo >actual &&
> + test_grep "create reason$" actual
> + )
> +'
> +
> +test_expect_success 'create with symref target creates target reference' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git symbolic-ref refs/heads/symref refs/heads/target &&
> + git refs create refs/heads/symref $A &&
> + git reflog exists refs/heads/target
> + )
> +'
> +
> +test_expect_success 'create with symref target and --no-deref refuses to create reference' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + git symbolic-ref refs/heads/symref refs/heads/target &&
> + test_must_fail git refs create --no-deref refs/heads/symref $A 2>err &&
> + test_grep "dangling symref already exists" err &&
> + test_must_fail git reflog exists refs/heads/target
> + )
> +'
Would it make sense to add this test:
test_expect_success 'create with symref target with --no-deref' '
test_when_finished "rm -rf repo" &&
setup_repo repo &&
(
cd repo &&
A=$(git rev-parse A) &&
git refs create refs/heads/target $A &&
git refs create --no-deref refs/heads/symref refs/heads/target &&
git reflog exists refs/heads/symref && false
)
'
But that makes me think, this option `--no-deref` is pretty obscure for
use with `git refs create`. There are two situations:
* The symref doesn't exists: so --no-deref basically is forcing the
command to create a symref. That's confusing
* The symref exists already: then the question is, does the user know it
exists:
- The user knows: so they pass --no-deref because they know it exists
and they want to create a symref. But why run `create` then anyway?
- The user doesn't know: brings us back to the first asterisk,
passing in `--no-deref` to create a symref, making it a weird
option name.
> +
> +test_expect_success 'create with empty message fails' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + (
> + cd repo &&
> + A=$(git rev-parse A) &&
> + test_must_fail git refs create --message= refs/heads/foo $A 2>err &&
> + test_grep "empty message" err &&
> + test_must_fail git refs exists refs/heads/foo
> + )
> +'
> +
> +test_expect_success 'create without arguments fails' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + test_must_fail git -C repo refs create 2>err &&
> + test_grep "requires reference name" err
> +'
> +
> +test_expect_success 'create with too many arguments fails' '
> + test_when_finished "rm -rf repo" &&
> + setup_repo repo &&
> + test_must_fail git -C repo refs create refs/heads/foo a b 2>err &&
> + test_grep "requires reference name" err
> +'
> +
> +test_done
>
> --
> 2.55.0.795.g602f6c329a.dirty
>
>
--
Cheers,
Toon
^ permalink raw reply
* Re: [PATCH v3 5/5] builtin/refs: add "rename" subcommand
From: Toon Claes @ 2026-07-03 14:31 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Junio C Hamano
In-Reply-To: <20260630-pks-refs-writing-subcommands-v3-5-deb04de1ecef@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
> Add a "rename" subcommand to git-refs(1) with the syntax:
>
> $ git refs rename <oldref> <newref>
>
> It renames <oldref> together with its reflog to <newref>; even when used
> on a local branch ref, the current value and the reflog of the ref are
> the only things that are renamed. Document it and redirect casual users
> to "git branch -m" if that is what they wanted to do.
>
> Co-authored-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> Documentation/git-refs.adoc | 6 ++
> builtin/refs.c | 49 +++++++++++++++++
> t/meson.build | 1 +
> t/t1467-refs-rename.sh | 131 ++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 187 insertions(+)
>
> diff --git a/Documentation/git-refs.adoc b/Documentation/git-refs.adoc
> index e6a3528349..ce278c59bf 100644
> --- a/Documentation/git-refs.adoc
> +++ b/Documentation/git-refs.adoc
> @@ -23,6 +23,7 @@ git refs optimize [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude
> git refs create [--message=<reason>] [--no-deref] [--create-reflog] <ref> <new-value>
> git refs delete [--message=<reason>] [--no-deref] <ref> [<old-value>]
> git refs update [--message=<reason>] [--no-deref] [--create-reflog] <ref> <new-value> [<old-value>]
> +git refs rename [--message=<reason>] <old-ref> <new-ref>
So symrefs cannot be renamed with this command?
--
Cheers,
Toon
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox