Reftable reflog timezone encoding differs from specification

Reftable reflog timezone encoding differs from specification

[Josh McKinney]

Hi,

Git 2.55.0 appears to store reftable reflog timezone offsets as signed
HHMM integers, whereas the specification requires signed minutes.

https://git-scm.com/docs/reftable#_log_record states:

    "tz_offset is the absolute number of minutes from GMT the
    committer was at the time of the update."

The specification also gives GMT+0230 as an example encoded as 150.

I reproduced the discrepancy on macOS arm64 by creating a SHA-1
reftable repository and committing with this date:

    2026-09-27T12:00:00+05:30

An independent Python/zlib inspection of the resulting reftable found:

    Stored timezone bytes: 02 12 = 530
    Expected signed minutes: 01 4a = 330

Both HEAD and refs/heads/main reflog entries contained 530. Git reads
its own entries back correctly as +05:30, so its writer and reader
appear internally consistent, but disagree with the specification.

Here is a reproducer using only Git and Python's standard library:

(
    set -eu
    proof_dir=$(mktemp -d)
    cd "$proof_dir"
    echo "Repository retained at: $proof_dir"

    export GIT_CONFIG_NOSYSTEM=1
    export GIT_CONFIG_GLOBAL=/dev/null
    export GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30'
    export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"

    git --version
    git init --initial-branch=main --object-format=sha1 \
        --ref-format=reftable example

    git -C example \
        -c user.name=Example \
        -c user.email=example@example.com \
        -c commit.gpgsign=false \
        -c core.logAllRefUpdates=true \
        commit --allow-empty -m "Timezone example"

    git -C example reflog show --format='%gD' \
        --date=iso-strict refs/heads/main

    python3 - <<'PY'
from pathlib import Path
import zlib

directory = Path("example/.git/reftable")
for name in (directory / "tables.list").read_text().splitlines():
    table = (directory / name).read_bytes()
    assert table[:5] == b"REFT\x01"
    footer = table[-68:]
    log_start = int.from_bytes(footer[48:56], "big")
    if not log_start:
        continue

    assert table[log_start:log_start + 1] == b"g"
    log = zlib.decompress(table[log_start + 4:])

    # Fixture-specific: email, variable-length time, then timezone.
    email = b"example@example.com"
    search_from = 0
    while (position := log.find(email, search_from)) != -1:
        position += len(email)
        while log[position] & 0x80:
            position += 1
        position += 1

        raw = log[position:position + 2]
        offset = int.from_bytes(raw, "big", signed=True)
        print(f"Timezone bytes: {raw.hex(' ')}; integer: {offset}")
        search_from = position + 2
PY
)

The relevant output is:

    refs/heads/main@{2026-09-27T12:00:00+05:30}
    Timezone bytes: 02 12; integer: 530
    Timezone bytes: 02 12; integer: 530

Is this a known discrepancy? Which representation should interoperable
implementations use? If either the implementation or specification
changes, how should existing tables be interpreted, given that values
such as 330 are valid under both interpretations?

Given that Git consistently writes and reads HHMM values, I suspect the practical resolution is to update the specification to match existing behavior. Are there other implementations or compatibility considerations that would prevent that?

Thanks,
Josh

-- 
Josh McKinney
joshka.net

Re: Reftable reflog timezone encoding differs from specification

[Patrick Steinhardt]

Hi,

On Mon, Sep 28, 2026 at 12:00:43AM -0700, Josh McKinney wrote:
> Hi,
> 
> Git 2.55.0 appears to store reftable reflog timezone offsets as signed
> HHMM integers, whereas the specification requires signed minutes.
> 
> https://git-scm.com/docs/reftable#_log_record states:
> 
>     "tz_offset is the absolute number of minutes from GMT the
>     committer was at the time of the update."
> 
> The specification also gives GMT+0230 as an example encoded as 150.

Oh dear, that's indeed the case. I was able to reproduce the issue, as
well.

> I reproduced the discrepancy on macOS arm64 by creating a SHA-1
> reftable repository and committing with this date:
> 
>     2026-09-27T12:00:00+05:30
> 
> An independent Python/zlib inspection of the resulting reftable found:
> 
>     Stored timezone bytes: 02 12 = 530
>     Expected signed minutes: 01 4a = 330

Yup. It's plain wrong the way we store it.

> Both HEAD and refs/heads/main reflog entries contained 530. Git reads
> its own entries back correctly as +05:30, so its writer and reader
> appear internally consistent, but disagree with the specification.
> 
> Here is a reproducer using only Git and Python's standard library:

And here's a Git reproducer:

diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c
index fc49fafc34..e3cfb6a3fa 100644
--- a/t/helper/test-reftable.c
+++ b/t/helper/test-reftable.c
@@ -103,7 +103,16 @@ static int dump_table(struct reftable_merged_table *mt)
 	if (err < 0)
 		return err;
 
-	algop = &hash_algos[hash_algo_by_id(reftable_merged_table_hash_id(mt))];
+	switch (reftable_merged_table_hash_id(mt)) {
+	case REFTABLE_HASH_SHA1:
+		algop = &hash_algos[GIT_HASH_SHA1];
+		break;
+	case REFTABLE_HASH_SHA256:
+		algop = &hash_algos[GIT_HASH_SHA256];
+		break;
+	default:
+		die("unsupported hash algorithm: %d", reftable_merged_table_hash_id(mt));
+	}
 
 	while (1) {
 		err = reftable_iterator_next_ref(&it, &ref);
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 35e98b43db..e1d714077c 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -1163,4 +1163,19 @@ test_expect_success 'writes do not persist peeled value for invalid tags' '
 	)
 '
 
+test_expect_success 'writes do not persist peeled value for invalid tags' '
+	test_when_finished rm -rf repo &&
+	git init repo &&
+	(
+		cd repo &&
+
+		export GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30' &&
+		export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" &&
+		git commit --allow-empty --message "Timezone example" &&
+		git refs optimize &&
+		test-tool dump-reftable -t .git/reftable/*.ref >table &&
+		test_grep "log{HEAD(2) C O Mitter <committer@example.com> 1790490600 0530" table
+	)
+'
+
 test_done

And yes, the test-helper for reftables is broken, so we also have to fix
that.

[snip]
> Is this a known discrepancy?

No, it's not, I wasn't aware of it at all.

> Which representation should interoperable implementations use?

We should use the one that we have in our specification, so in my
opinion we should fix Git itself. This is also because JGit, which had a
reftable implementation for far longer compared to us, implements the
specification correctly:

	private PersonIdent readPersonIdent() {
		String name = readValueString();
		String email = readValueString();
		long epochSeconds = readVarint64();
		ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);
		return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);
	}

You can see that we indeed treat the integer as number of minutes there,
as expected.

> If either the implementation or specification changes, how should
> existing tables be interpreted, given that values such as 330 are
> valid under both interpretations?
> 
> Given that Git consistently writes and reads HHMM values, I suspect
> the practical resolution is to update the specification to match
> existing behavior. Are there other implementations or compatibility
> considerations that would prevent that?

That's a very good question. Given that JGit interprets the value as
expected my take is that we should fix this in Git and keep the spec
as-is.

The question is how much we really lose by "just" fixing the bug. Sure,
timezones would be wrong in that case. For example, if we had an entry
with original timezone of +0200 we'd now interpret that as +0320. It's
of course wrong given the original intent, but is it the end of the
world? I dunno. Overall, the amount of damage is kind of limited here as
the discrepancy is limited:

  ┌──────┬──────────────┬────────────────┬───────────┐
  │tz    │HHMM encoding │correct minutes │divergence │
  ├──────┼──────────────┼────────────────┼───────────┤
  │+1400 │1400          │840             │560        │
  ├──────┼──────────────┼────────────────┼───────────┤
  │-1200 │-1200         │-720            │480        │
  ├──────┼──────────────┼────────────────┼───────────┤
  │+0530 │530           │330             │200        │
  ├──────┼──────────────┼────────────────┼───────────┤
  │+0000 │0             │0               │0          │
  └──────┴──────────────┴────────────────┴───────────┘

We could of course retroactively declare that version 2 of the format
uses the syntax that Git uses right now. After all, JGit only knows to
read version 1 of it anyway, so that could kind of fix it. But for any
repository that uses SHA1 we used to write version 1 anyway, so this
does not really buy us anything, I'd claim.

In summary:

  - We have an upper limit in divergence of <10h.

  - This only matters in the context of reflogs, we don't use these
    anywhere else.

  - The risk for data loss by a change is limited as our default grace
    period for garbage collecting reflog entries is 30 days.

With these points I'm inclined to call it a bug and just fix it, without
handling backwards compatibility.

I'm very happy to hear alternative takes though.

In any case, thanks for your report!

Patrick

Re: Reftable reflog timezone encoding differs from specification

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> We should use the one that we have in our specification, so in my
> opinion we should fix Git itself. This is also because JGit, which had a
> reftable implementation for far longer compared to us, implements the
> specification correctly:
>
>
> 	private PersonIdent readPersonIdent() {
> 		String name = readValueString();
> 		String email = readValueString();
> 		long epochSeconds = readVarint64();
> 		ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);
> 		return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);
> 	}

Thanks for checking.  I (unfortunately) agree with the (unfortunate)
conclusion.

We do not ship reftable files over networks and reflogs at the
conceptual level is not shared across repositories, so the issue,
other than the trivial part of updating the implementation, is how
to migrate the data in a local repository that uses reftable.  One
time offline conversion may be the simplest but I do not know if it
is worth it, given ...

> We could of course retroactively declare that version 2 of the format
> uses the syntax that Git uses right now. After all, JGit only knows to
> read version 1 of it anyway, so that could kind of fix it. But for any
> repository that uses SHA1 we used to write version 1 anyway, so this
> does not really buy us anything, I'd claim.
>
> In summary:
>
>   - We have an upper limit in divergence of <10h.

... this.

>
>   - This only matters in the context of reflogs, we don't use these
>     anywhere else.
>
>   - The risk for data loss by a change is limited as our default grace
>     period for garbage collecting reflog entries is 30 days.
>
> With these points I'm inclined to call it a bug and just fix it, without
> handling backwards compatibility.

Re: Reftable reflog timezone encoding differs from specification

[Josh McKinney]

I think my main concern here is mostly around using multiple tools on the same repo and how they should interpret on disk formats (my clanker picked up the problem when comparing git's output with a library it's writing).

Anyway, nothing urgent on the problem from me because I noticed it purely in a development context.
Thanks for filling in the bits about the real world impact on this too.

Josh

-- 
Josh McKinney
joshka.net

On Mon, Sep 28, 2026, at 7:42 AM, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
>> We should use the one that we have in our specification, so in my
>> opinion we should fix Git itself. This is also because JGit, which had a
>> reftable implementation for far longer compared to us, implements the
>> specification correctly:
>>
>>
>> 	private PersonIdent readPersonIdent() {
>> 		String name = readValueString();
>> 		String email = readValueString();
>> 		long epochSeconds = readVarint64();
>> 		ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);
>> 		return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);
>> 	}
>
> Thanks for checking.  I (unfortunately) agree with the (unfortunate)
> conclusion.
>
> We do not ship reftable files over networks and reflogs at the
> conceptual level is not shared across repositories, so the issue,
> other than the trivial part of updating the implementation, is how
> to migrate the data in a local repository that uses reftable.  One
> time offline conversion may be the simplest but I do not know if it
> is worth it, given ...
>
>> We could of course retroactively declare that version 2 of the format
>> uses the syntax that Git uses right now. After all, JGit only knows to
>> read version 1 of it anyway, so that could kind of fix it. But for any
>> repository that uses SHA1 we used to write version 1 anyway, so this
>> does not really buy us anything, I'd claim.
>>
>> In summary:
>>
>>   - We have an upper limit in divergence of <10h.
>
> ... this.
>
>>
>>   - This only matters in the context of reflogs, we don't use these
>>     anywhere else.
>>
>>   - The risk for data loss by a change is limited as our default grace
>>     period for garbage collecting reflog entries is 30 days.
>>
>> With these points I'm inclined to call it a bug and just fix it, without
>> handling backwards compatibility.

Re: Reftable reflog timezone encoding differs from specification

[Patrick Steinhardt]

On Mon, Sep 28, 2026 at 09:02:36AM -0700, Josh McKinney wrote:
> I think my main concern here is mostly around using multiple tools on
> the same repo and how they should interpret on disk formats (my
> clanker picked up the problem when comparing git's output with a
> library it's writing).

Yeah, and sticking to the spec we have is the best way to fix that, I'd
think.

> Anyway, nothing urgent on the problem from me because I noticed it
> purely in a development context. Thanks for filling in the bits about
> the real world impact on this too.

Well, I think fixing it is somewhat urgent -- the longer we have the
inconsistency the more problems it causes. I'll aim for having a fix for
this ready later this week.

Thanks for detecting this!

Patrick