Signed commit regression?

Signed commit regression?

[Linus Torvalds]

I suspect this may be due to gpg updates, not git, but I'm about to
leave for the airport in an hour and don't have time to look into it
more closely right now.

Because of the imminent travel, I did a "git pull" on my laptop, and
it doesn't have all the pgp keys I have on my desktop. It was a signed
tag, but the pull results in:

  commit bfeb4f9977348daaaf7283ff365d81f7ee95940a
  merged tag 'zonefs-5.6-rc4'
  No signature
  Merge: 45d0b75b98bf 0dda2ddb7ded
  Author: Linus Torvalds <torvalds@linux-foundation.org>
  Date:   5 minutes ago

      Merge tag 'zonefs-5.6-rc4' of
git://git.kernel.org/pub/scm/linux/kernel/git/dlemoal/zonefs

      Pull zonefs fixes from Damien Le Moal:
       "Two fixes in here:
    ....

and notice the "No signature". It's entirely wrong. There _is_ a
signature, it's just that we don't have the key.

And the "No signature" thing is particularly unhelpful, because it now
doesn't show us what key is missing, like it used to.

"git verify-tag" still works correctly:

  [torvalds@xps13 linux]$ git verify-tag FETCH_HEAD
  gpg: Signature made Fri 28 Feb 2020 12:03:36 AM PST
  gpg:                using EDDSA key 913EFF2D612BE1C00CC97738DDA1CDD2C5DA1876
  gpg: Can't check signature: No public key

and shows the key ID, and properly says "Can't check signature" (which
is very very different from "No signature").

This is a big regression. The "No signature" message really is
completely incorrect, and is very very wrong indeed.

I suspect it's due to this commit:

  72b006f4bf ("gpg-interface: prefer check_signature() for GPG verification")

but as mentioned I don't have the ability to really dig deeper right now.

                Linus

Re: Signed commit regression?

[Linus Torvalds]

On Fri, Feb 28, 2020 at 8:44 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> I suspect it's due to this commit:
>
>   72b006f4bf ("gpg-interface: prefer check_signature() for GPG verification")
>
> but as mentioned I don't have the ability to really dig deeper right now.

Never mind - I did a mindless "just revert that and test", and it
indeed is that commit.

Please revert it in upstream git. The "No signature" message really is
horribly wrong. It's both technically entirely wrong, but it's wrong
from a UI standpoint too since you really need to show what the
missing key was:

With the revert, I get the proper output:

  commit bfeb4f9977348daaaf7283ff365d81f7ee95940a
  merged tag 'zonefs-5.6-rc4'
  gpg: Signature made Fri 28 Feb 2020 12:03:36 AM PST
  gpg:                using EDDSA key 913EFF2D612BE1C00CC97738DDA1CDD2C5DA1876
  gpg: Can't check signature: No public key
  Merge: 45d0b75b98bf 0dda2ddb7ded
  Author: Linus Torvalds <torvalds@linux-foundation.org>
  Date:   10 minutes ago

      Merge tag 'zonefs-5.6-rc4' of
git://git.kernel.org/pub/scm/linux/kernel/git/dlemoal/zonefs

      Pull zonefs fixes from Damien Le Moal:
       "Two fixes in here:
       ....

which is useful.

                    Linus

Re: Signed commit regression?

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Fri, Feb 28, 2020 at 8:44 AM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> I suspect it's due to this commit:
>>
>>   72b006f4bf ("gpg-interface: prefer check_signature() for GPG verification")
>>
>> but as mentioned I don't have the ability to really dig deeper right now.
>
> Never mind - I did a mindless "just revert that and test", and it
> indeed is that commit.
>
> Please revert it in upstream git. The "No signature" message really is
> horribly wrong. It's both technically entirely wrong, but it's wrong
> from a UI standpoint too since you really need to show what the
> missing key was.

True---the messages that told you the missing piece of information
with the original code came directly from gnupg, and the problematic
change stopped showing that and replaced it with the generic (and
wrong) "We tried to verify signature and it failed---it must be that
the input did not have signature" message.

It is in v2.25 already, so we'd need to revert it out of 'maint'; it
seems to have a minimum fallout on a topic in flight, but it looks
manageable.

Thanks.

Re: Signed commit regression?

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
>> On Fri, Feb 28, 2020 at 8:44 AM Linus Torvalds
>> <torvalds@linux-foundation.org> wrote:
>>>
>>> I suspect it's due to this commit:
>>>
>>>   72b006f4bf ("gpg-interface: prefer check_signature() for GPG verification")
>>>
>>> but as mentioned I don't have the ability to really dig deeper right now.
>>
>> Never mind - I did a mindless "just revert that and test", and it
>> indeed is that commit.
>>
>> Please revert it in upstream git. The "No signature" message really is
>> horribly wrong. It's both technically entirely wrong, but it's wrong
>> from a UI standpoint too since you really need to show what the
>> missing key was.
>
> True---the messages that told you the missing piece of information
> with the original code came directly from gnupg, and the problematic
> change stopped showing that and replaced it with the generic (and
> wrong) "We tried to verify signature and it failed---it must be that
> the input did not have signature" message.
>
> It is in v2.25 already, so we'd need to revert it out of 'maint'; it
> seems to have a minimum fallout on a topic in flight, but it looks
> manageable.

I've prepared a topic to revert that commit and it is now in the
middle of 'pu'; it will get merged down to 'next', 'master' and
'maint' in due course as other topics.

Brian's SHA-256 (1/4) topic had a couple of changes that depended on
the GPG interface API from 72b006f4 ("gpg-interface: prefer
check_signature() for GPG verification", 2019-11-27), so I ejected
them out of the topic for now:

    - tag: store SHA-256 signatures in a header
    - gpg-interface: improve interface for parsing tags

In the longer term, however, we do want an updated GPG interface
layer that lets us achieve 72b006f4 wanted to, namely

 - have a single entry point into GPG interface API, so that the
   changes in the future can be centralized;

 - not to depend _too_ heavily on the GnuPG's behaviour.  The pieces
   of information that was lost from our output and made Linus upset
   was given to the end-user by us parrotting what gpg said without
   the code really understanding what is being said, and we should
   instead make our code aware of _why_ verify_signed_buffer() or
   check_signature() have failed, and make sure we report that to
   the callers.

I'd expect that there may be another round of attempt to update the
GPG interface.  Let's make sure we won't lose info given to the
end-users while doing so.

Thanks.

Re: Signed commit regression?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

On 2020-02-28 at 18:24:33, Junio C Hamano wrote:
> I've prepared a topic to revert that commit and it is now in the
> middle of 'pu'; it will get merged down to 'next', 'master' and
> 'maint' in due course as other topics.
> 
> Brian's SHA-256 (1/4) topic had a couple of changes that depended on
> the GPG interface API from 72b006f4 ("gpg-interface: prefer
> check_signature() for GPG verification", 2019-11-27), so I ejected
> them out of the topic for now:
> 
>     - tag: store SHA-256 signatures in a header
>     - gpg-interface: improve interface for parsing tags

That's fine.  I'll rebase my series on top of that patch.  Thanks for
letting me know.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: Signed commit regression?

[Hans Jerry Illikainen]

On Fri, Feb 28 2020, Junio C Hamano wrote:
> I'd expect that there may be another round of attempt to update the
> GPG interface.  Let's make sure we won't lose info given to the
> end-users while doing so.

The regression was introduced by this botched chunk in 72b006f4bf:

@@ -521,18 +522,19 @@ static int show_one_mergetag(struct commit *commit,
 	gpg_message_offset = verify_message.len;

 	payload_size = parse_signature(extra->value, extra->len);
 	status = -1;
 	if (extra->len > payload_size) {
 		/* could have a good signature */
-		if (!verify_signed_buffer(extra->value, payload_size,
-					  extra->value + payload_size,
-					  extra->len - payload_size,
-					  &verify_message, NULL))
+		if (!check_signature(extra->value, payload_size,
+				     extra->value + payload_size,
+				     extra->len - payload_size, &sigc)) {
+			strbuf_addstr(&verify_message, sigc.gpg_output);
+			signature_check_clear(&sigc);
 			status = 0; /* good */
-		else if (verify_message.len <= gpg_message_offset)
+		} else if (verify_message.len <= gpg_message_offset)
 			strbuf_addstr(&verify_message, "No signature\n");
 		/* otherwise we couldn't verify, which is shown as bad */
 	}

There are two ridiculous bugs in my original patch:

1. The output from GPG is only added to `verify_message' if a signature
   verifies successfully.

2. On verification failure, the "No signature" message is always added
   to `verify_message'.  This is because, again, no output from GPG is
   added to `verify_message' on failure, so its length will always equal
   `gpg_message_offset' (see the initial assignment) when
   `check_signature()' returns non-zero, sigh...

Not sure of the proper way of fixing a reverted commit, but I'll send v1
based on pu that includes regression tests.

I'm sorry for my fuckup and the headache it caused!

-- 
hji