ref-format checking regression

ref-format checking regression

[Jeff King]

I upgraded git on a machine recently, and it created problems for a repo
with a bogus character in a ref name.  Older versions of git never
complained about it. Newer ones, containing your dce4bab ("add_ref():
verify that the refname is formatted correctly") do. That's fine; it's
bogus and git _should_ complain about it.

However, recovering from the situation is unnecessarily hard, as basic
things like fsck stop working, you can't access the ref via rev-parse,
etc. You can reproduce with something like this (in some repo with
actual commits):

  $ evil=$(printf 'foo\177bar')
  $ git rev-parse HEAD >".git/refs/tags/$evil"

  $ git fsck
  fatal: Reference has invalid format: 'refs/tags/foo^?bar'

  [does not even warn about the real reason for the error]
  $ git rev-parse --verify "$evil"
  fatal: Needed a single revision

  [does not let you find broken refs]
  $ git for-each-ref
  fatal: Reference has invalid format: 'refs/tags/foo^?bar'
  $ git tag -l
  fatal: Reference has invalid format: 'refs/tags/foo^?bar'

  [no way to rename or delete the bogus tag]
  $ git tag fixed "$evil"
  fatal: Failed to resolve 'foo^?bar' as a valid ref.

I seem to recall discussing this format-tightening and trying to be sure
that users were left with a way forward for fixing their repos. But I
can't find the discussion, and I don't recall any conclusion we came to.
So maybe we decided not to worry about it. But I thought I'd mention it
as a data point.

-Peff

Re: ref-format checking regression

[Ævar Arnfjörð Bjarmason]

On Fri, Apr 27, 2012 at 13:50, Jeff King <peff@peff.net> wrote:
> I seem to recall discussing this format-tightening and trying to be sure
> that users were left with a way forward for fixing their repos. But I
> can't find the discussion, and I don't recall any conclusion we came to.
> So maybe we decided not to worry about it. But I thought I'd mention it
> as a data point.

Wasn't that the one about the tightening of the ref checks breaking
git clone --mirror?

In any case we should be strict on input and loose when going over
existing data. It seems to me that we're using one function now to
check the validity of refs when we should be using two, one for new
refs, and one for reading existing data.

Re: ref-format checking regression

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I upgraded git on a machine recently, and it created problems for a repo
> with a bogus character in a ref name.  Older versions of git never
> complained about it. Newer ones, containing your dce4bab ("add_ref():
> verify that the refname is formatted correctly") do. That's fine; it's
> bogus and git _should_ complain about it.
> 
> However, recovering from the situation is unnecessarily hard, ...
> ...
> I seem to recall discussing this format-tightening and trying to be sure
> that users were left with a way forward for fixing their repos. But I
> can't find the discussion, and I don't recall any conclusion we came to.

I haven't dug the archive but I do recall pointing many issues out
around the theme "be liberal in what you accept and strict in what you
produce" on this topic, and loosening one or two showstoppers during the
review cycle, but obviously we did not catch all of them.

Michael?

Re: ref-format checking regression

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jeff King <peff@peff.net> writes:
>
>> I upgraded git on a machine recently, and it created problems for a repo
>> with a bogus character in a ref name.  Older versions of git never
>> complained about it. Newer ones, containing your dce4bab ("add_ref():
>> verify that the refname is formatted correctly") do. That's fine; it's
>> bogus and git _should_ complain about it.
>> 
>> However, recovering from the situation is unnecessarily hard, ...
>> ...
>> I seem to recall discussing this format-tightening and trying to be sure
>> that users were left with a way forward for fixing their repos. But I
>> can't find the discussion, and I don't recall any conclusion we came to.
>
> I haven't dug the archive but I do recall ...

A few...

  http://thread.gmane.org/gmane.comp.version-control.git/185564
  http://thread.gmane.org/gmane.comp.version-control.git/184382/focus=185483

Re: ref-format checking regression

[Jeff King]

On Fri, Apr 27, 2012 at 08:06:26AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > I upgraded git on a machine recently, and it created problems for a repo
> > with a bogus character in a ref name.  Older versions of git never
> > complained about it. Newer ones, containing your dce4bab ("add_ref():
> > verify that the refname is formatted correctly") do. That's fine; it's
> > bogus and git _should_ complain about it.
> > 
> > However, recovering from the situation is unnecessarily hard, ...
> > ...
> > I seem to recall discussing this format-tightening and trying to be sure
> > that users were left with a way forward for fixing their repos. But I
> > can't find the discussion, and I don't recall any conclusion we came to.
> 
> I haven't dug the archive but I do recall pointing many issues out
> around the theme "be liberal in what you accept and strict in what you
> produce" on this topic, and loosening one or two showstoppers during the
> review cycle, but obviously we did not catch all of them.

I should point out that this was due to GitHub recently upgrading the
version of git on our backend servers. And out of the bazillion repos we
host, I have so far seen only one actual bug report. So while it might
be nice to be more friendly, it may simply not be all that common an
issue (and in this case, we were able to resolve it manually). If it's
easy to fix, I think we should, but if the fix ends up being very
complex, it might not be worth the trouble.

-Peff