[Survey] Signed push

[Survey] Signed push

[Junio C Hamano]

[administrivia] This message is also Cc'ed to the kernel mailing list in
order to ask for opinions from members of one of the most important user
communities of Git, but people may want to drop the kernel list when
responding to this message to reduce the noise level over there. Thanks.

In the light of what happened to k.org recently, we've been discussing
things Git can do to help raising confidence levels perceived by the
general public on integrity of the source trees, especially for the kernel
community. As the article by Jonathan Corbet on lwn.net nicely described,
projects managed with Git are already pretty resistant from tampering, and
it is not my (nor anybody in the Git community's) intention to propose any
more unnecessary bureaucracy to the development process without merit.

There are two updates that may change the end user experience I would like
to ask your opinions on, both as the Git designer (emeritus?) and as the
top kernel developer.


1. Improved pull requests.

Currently a typical pull-request begins like this:

    The following changes since commit f696543dad6c7ba27b0c4fab167a5687263a9ba0:

      Flobar 2.4.3 (2011-09-13 12:34:56 +0900)

    are available in the git repository at:
      git://git.kernel.org/pub/flobar.git/ master

which is followed by the shortlog and expected diffstat.  This tells you
where the requester based his work on in excruciating detail, but does not
tell you what you should expect to fetch, any more than "whatever happened
to be at the named branch when you happened to notice the request."

We have a tentative patch to add an extra line after the "URL branch" line
that is for your cut & paste that looks like:

    are available in the git repository at:
      git://git.kernel.org/pub/flobar.git/ master
    for you to fetch changes up to 5738c9c21e53356ab5020912116e7f82fd2d428f

I often see you respond to a pull request on the kernel mailing list with
"I see nothing new; forgot to push?", and having this extra line may also
help communication.

Would it be just an added and useless noise that you nor your requesters
would not care much about?

An alternative that I am considering is to let the requester say this
instead:

    are available in the git repository at:
      git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f

without adding the extra line.

That is, to allow fetching the history up to an explicitly named commit
object. This would only involve a change to fetch-pack at the receiving
end; just match the commit object name given from the command line against
the ls-remote response and ask upload-pack to give the history leading to
it. The released versions of Git already will happily oblige, as long as
the commit object named in the request message still sits at the tip of
the intended branch.

Do you think it is worthwhile to pursue this alternative?


2. Signed pushes.

You tag official releases and release candidates with your GPG key, and
everybody who works within the kernel ecosystem trusts the history behind
the commits pointed by them, but there is no easy way to verify that
commits and merges between the last tagged commit and the tip of your
branch(es) are indeed from you, or if an intruder piled fake ones on top
of your commits (until you try to push again and discover that the history
does not fast-forward, that is).

We have been discussing an addition of "git push -s" to let people sign
their pushes (instead of having to sign every commit or add signed
tag). The implementation alternatives were being bikeshed but not of much
interest in this message, but the user experience would go like this:

 * You push out your work with "git push -s";

 * "git push" prepares a "push certificate" (it is meant to certify "these
   are the commits I place at the tips of these refs"), which is a human
   and machine readable text file in core, that may look like this:

        Push-Certificate-Version: 0
        Pusher: Junio C Hamano <gitster@pobox.com>
        Update: 3793ac56b4c4f9bf0bddc306a0cec21118683728 refs/heads/master
        Update: 12850bec0c24b529c9a9df6a95ad4bdeea39373e refs/heads/next

   and asks you to GPG sign it. You only unlock your GPG key and the
   command internally runs GPG, just like "tag -s".

 * When "git push" finishes, the receiving end has this record in its
   refs/notes/signed-push notes tree, together with your previous pushes
   (as this is not a shared repository, it will record only your pushes).
   The notes annnotate the commits named on the "Update:" lines above.

 * People who want to verify commits that are not yet tagged near the tip
   in their clone of your tree can fetch refs/notes/signed-push and run

     $ git log --show-notes=signed-push --branches --not --tags

   to see your push certificates as annotations on commits that are not
   yet tagged. They can verify them using a tool (yet to be written) that
   acts like "git tag --verify".

It is hoped that it would help downstream with warm and fuzzy assurances
that all commits including the ones that are not yet tagged are genuine
(disclaimer: my employer is among the "downstream" that wants to have that
warm and fuzzy assurance) if we can see these push certificates published
at your public repository.

A few questions.

 * As a user, do you think "signed push" is a good idea, or is it merely
   an unnecessary bureaucracy, having to sign all pushes?

 * As a user, do you think it is a good thing that you could also verify
   the commits you receive from the Git-managed repositories of your
   lieutenants using this mechanism, or you wouldn't bother, perhaps
   because you are applying many patches sent via unsigned e-mail from
   Andrew anyway?

 * If the answers to the above points are both "yes", do you think it
   would make sense to also propagate the push certificates you obtain
   from your lieutenants to your public repository when you make your
   "push -s"? It will allow your downstream to follow the chain of trust
   in one-go (if you are pulling from public places, they can fetch the
   push certificates from your lieutenants themselves and merge them, so
   this is merely a convenience feature) by simply fetching from the
   refs/notes/signed-push notes tree from your public repository.  Do you
   think it is a useful and worthwhile thing to do?

Re: [Survey] Signed push

[Sam Vilain]

On 9/13/11 9:45 AM, Junio C Hamano wrote:
>   * You push out your work with "git push -s";
>
>   * "git push" prepares a "push certificate" (it is meant to certify "these
>     are the commits I place at the tips of these refs"), which is a human
>     and machine readable text file in core, that may look like this:
>
>          Push-Certificate-Version: 0
>          Pusher: Junio C Hamano<gitster@pobox.com>
>          Update: 3793ac56b4c4f9bf0bddc306a0cec21118683728 refs/heads/master
>          Update: 12850bec0c24b529c9a9df6a95ad4bdeea39373e refs/heads/next
>
>     and asks you to GPG sign it. You only unlock your GPG key and the
>     command internally runs GPG, just like "tag -s".
>
>   * When "git push" finishes, the receiving end has this record in its
>     refs/notes/signed-push notes tree, together with your previous pushes
>     (as this is not a shared repository, it will record only your pushes).
>     The notes annnotate the commits named on the "Update:" lines above.

If the push certificate also has the previous commit IDs for the changed 
refs, then you actually have an audit log.  Otherwise, it does not 
certify the commit range they pushed.

This is an important prerequisite for a fully distributed, peer to peer 
git.  For this case it would also need something to distinguish which 
repository is to be updated; such as a canonical repository URL (or list 
of URLs), or just a short project name.  A P2P protocol can then know 
projects as (KEYID, projectname).

Sam

Fwd: [Survey] Signed push

[Linus Torvalds]

Recovering lost emails. Or maybe you get duplicates. Sorry about that if so,

                   Linus

---------- Forwarded message ----------
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Tue, Sep 13, 2011 at 10:48 AM
Subject: Re: [Survey] Signed push
To: Junio C Hamano <gitster@pobox.com>
Cc: git@vger.kernel.org, linux-kernel@vger.kernel.org


On Tue, Sep 13, 2011 at 9:45 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> We have a tentative patch to add an extra line after the "URL branch" line
> that is for your cut & paste that looks like:
>
>    are available in the git repository at:
>      git://git.kernel.org/pub/flobar.git/ master
>    for you to fetch changes up to 5738c9c21e53356ab5020912116e7f82fd2d428f
>
> I often see you respond to a pull request on the kernel mailing list with
> "I see nothing new; forgot to push?", and having this extra line may also
> help communication.

I think that would probably be a good idea, although I'd actually
prefer you to be more verbose, and more human-friendly, and actually
talk about the commit in a readable way. Get rid of the *horrible*
BRANCH-NOT-VERIFIED message (that actually messes up pull requests if
mirroring is a bit delayed and throws away more important
information), and instead just have a blurb afterwards saying
something human-readable like

 Top commit 1f51b001cccf: "Merge branches 'cns3xxx/fixes',
 'omap/fixes' and 'davinci/fixes' into fixes"

 and at *that* point you might have a "UNVERIFIED" notice for people
to check if they forgot to push.

So I'd much prefer something like that over:

> An alternative that I am considering is to let the requester say this
> instead:
>
>    are available in the git repository at:
>      git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
>
> without adding the extra line.

The extra line in the pull request is cheap - it's not like we need to
ration them. The above format, in contrast, requires that the person
doing the *pull* have a recent enough git client, otherwise the merge
commit message will be just horrible.

And even if you do have a new git client that turns the commit into a
branch name, that's ambigious. What if both 'master' and
'experimental' have the same top commit, because experimental ended up
being tested and was percolated to master? Which branch name would you
pick? And what if the branch was updated since, so *no* branch name
matches - does that mean that you'd disallow the pull entirely?

> 2. Signed pushes.
>
> You tag official releases and release candidates with your GPG key, and
> everybody who works within the kernel ecosystem trusts the history behind
> the commits pointed by them, but there is no easy way to verify that
> commits and merges between the last tagged commit and the tip of your
> branch(es) are indeed from you, or if an intruder piled fake ones on top
> of your commits (until you try to push again and discover that the history
> does not fast-forward, that is).
>
> We have been discussing an addition of "git push -s" to let people sign
> their pushes (instead of having to sign every commit or add signed
> tag). The implementation alternatives were being bikeshed but not of much
> interest in this message, but the user experience would go like this:

Also, if we're adding branch information, I'd say that a description
of the branch is more important than a signature. Right now we lack
even that.

It would be lovely if people could annotate their branches with
descriptions, so that when I pull a "for-linus" branch, if it has a
description, the description of the branch makes it into the merge
message. Our merge messages are often not very informative.

I realize that cryptographic signature sound very important right now,
but in the end, *real* trust comes from people, not from signatures.
Realistically, I checked a few signatures this time around due to the
k.org issues, but at the same thing, the thing that made me trust most
of it was just looking at commits and the email messages. The
unconscious and non-cryptographic "signature" of a person acting like
you expect a person to act.

Technical measures can be subverted, and I think we should also think
about the social side. Every time somebody mentions a signature, I
want to also mention "human readability", because I think that matters
as much, if not more.

So I'm not against signed pushes, but quite frankly, if you add some
per-branch signature, I would argue against it unless that signature
also comes with information that allows us to do a better job of human
communication too. Like a branch description.

Imagine, for example, than when you do a

  git push -s ..

git would *require* you to actually write a message about what you are
pushing. And when somebody pulls it, and creates a merge commit, that
explanation would become part of the merge message. The "signature"
part of the "-s" should be thought of as the *much* less interesting
part - that's just a small detail that git can use to verify
something, but it doesn't actually matter for the contents of the
pull. Not like the actual human-readable message would.

Now *that* would be lovely. No?

                       Linus

Re: [Survey] Signed push

[Andy Lutomirski]

On 09/13/2011 09:45 AM, Junio C Hamano wrote:
> 
> An alternative that I am considering is to let the requester say this
> instead:
> 
>     are available in the git repository at:
>       git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
> 
> without adding the extra line.
> 
> That is, to allow fetching the history up to an explicitly named commit
> object. This would only involve a change to fetch-pack at the receiving
> end; just match the commit object name given from the command line against
> the ls-remote response and ask upload-pack to give the history leading to
> it. The released versions of Git already will happily oblige, as long as
> the commit object named in the request message still sits at the tip of
> the intended branch.

I would love this feature on the pull/fetch interface, but for a
completely different reason.  Sometimes I want to pull a particular
object (usually a commit, but sometimes just a tree or blob) from
*myself*, and having to stick it on a branch is annoying.

One use-case is when applying a patch in git's extended format.  If I
know where it came from, I ought to be able to pull the blobs it depends
on to enable three-way merge.  I think that this is essentially
impossible remotely right now.

Of course, merging with the result of the pull will result in terrible
automatically-generated messages, but it's easy to fix that up manually.

This is one thing that I think Mercurial handles better than git.  (And
apologies for the noise if I've missed a way to do this with current
git.  I've looked, but maybe I missed some magic way to do this.)

--Andy