Deleting of the specified ref during the post-receive hook

Deleting of the specified ref during the post-receive hook

[Joshua Jensen]

  I've recently asked whether there was a public script to act as an 
automatic 'maintainer', something akin to Gerrit's push always 
succeeding without having to pull first when using Git as a central 
repository.  I received a number of suggestions, and I have begun trial 
implementations.

My current line of thought has an auto-merging script that monitors the 
refs/for/ namespace (similar to Gerrit) and then applies --no-ff merges 
to the appropriate branch.  For instance, when the user pushes to 
refs/for/master, the post-receive hook creates a secondary ref called 
refs/for/master-SHA1-timestamp and then deletes the refs/for/master ref:

#!bin/sh
# post-receive hook
while read oldrev newrev ref
do
     case $ref in
         refs/for/*)
             timestamp=`date +%s`
             `git update-ref $ref-$newrev-$timestamp $newrev`
             `git update-ref -d $ref`
         ;;
     esac
done

If you'll pardon my lacking shell script skills (I'm open to learn!), my 
primary question concerns safety.  When receiving a ref via an SSH-based 
server (which happens to be Gitolite, but I don't think that is relevant 
here), is the post-receive hook guaranteed to be run in a lockstep 
manner?  That is, if two people push to 'refs/for/master' at the same 
time, is there a lock to process one user and then the other user?

The auto-merging script is just simple at the moment.  It runs 'git 
fetch origin refs/for/*:refs/for/*', sorts the refs/for/ entries by 
timestamp, and merges into the specified branch, emailing the user on 
success or failure (not implemented yet... I'm sure Gitolite gives me 
access to the username, but I haven't looked it up yet).

Before I go too much deeper down this path, am I way off base here?

Thanks.

Josh

Re: Deleting of the specified ref during the post-receive hook

[Shawn O. Pearce]

Joshua Jensen <jjensen@workspacewhiz.com> wrote:
>  I've recently asked whether there was a public script to act as an  
> automatic 'maintainer', something akin to Gerrit's push always  
> succeeding without having to pull first when using Git as a central  
> repository.  I received a number of suggestions, and I have begun trial  
> implementations.
>
> My current line of thought has an auto-merging script that monitors the  
> refs/for/ namespace (similar to Gerrit) and then applies --no-ff merges  
> to the appropriate branch.  For instance, when the user pushes to  
> refs/for/master, the post-receive hook creates a secondary ref called  
> refs/for/master-SHA1-timestamp and then deletes the refs/for/master ref:
>
> #!bin/sh
> # post-receive hook
> while read oldrev newrev ref
> do
>     case $ref in
>         refs/for/*)
>             timestamp=`date +%s`
>             `git update-ref $ref-$newrev-$timestamp $newrev`
>             `git update-ref -d $ref`

FWIW, you don't need `` around the git update-ref calls, because you
aren't using the output of the command as input to another command.

> If you'll pardon my lacking shell script skills (I'm open to learn!), my  
> primary question concerns safety.  When receiving a ref via an SSH-based  
> server (which happens to be Gitolite, but I don't think that is relevant  
> here), is the post-receive hook guaranteed to be run in a lockstep  
> manner?

No.  If there are two concurrent pushes occurring, the script may
execute in parallel.

But you'll actually get something much worse.  receive-pack will
create refs/for/master for the first user... and then might be put
to sleep while another user's receive-pack starts.  That second
user will see refs/for/master existing, and will fail their push
because their concept of $newrev doesn't match what is currently
at refs/for/master.  Then the first user wakes up, runs your
post-receive, and the ref is cleared.

The better strategy would be to use an update hook that refuses to
permit the creation of refs/for/master:

  #!/bin/sh
  ref=$1
  old=$2
  new=$3

  case $ref in
  refs/for/*)
    timestamp=`date +%s`
    git update-ref $ref-$new-$timestamp $new
    echo "Created $ref-$new-$timestamp"
    exit 1
    ;;
  *)
    exit 0
    ;;
  esac


By exit 1 here we refuse the push attempt, so receive-pack won't
create refs/for/master, and another user pushing won't see that
false failure.  However, unlike with Gerrit, every user is now going
to receive a push failure message because the hook has appeared to
reject the value, even though it accepted it.

> That is, if two people push to 'refs/for/master' at the same  
> time, is there a lock to process one user and then the other user?

There is, but only for the duration of the update hook.  So if you
used an update hook instead, one user would succeed with the push
and the other would get a lock failure, and would need to retry.

> Before I go too much deeper down this path, am I way off base here?

I'd have to ask why you are using gitolite and trying to abuse
git-receive-pack to do something that Gerrit does out of the box...

-- 
Shawn.

Re: Deleting of the specified ref during the post-receive hook

[Joshua Jensen]

  ----- Original Message -----
From: Shawn O. Pearce
Date: 9/21/2010 8:50 AM
> Joshua Jensen<jjensen@workspacewhiz.com>  wrote:
>> My current line of thought has an auto-merging script that monitors the
>> refs/for/ namespace (similar to Gerrit) and then applies --no-ff merges
>> to the appropriate branch.  For instance, when the user pushes to
>> refs/for/master, the post-receive hook creates a secondary ref called
>> refs/for/master-SHA1-timestamp and then deletes the refs/for/master ref
>>
> FWIW, you don't need `` around the git update-ref calls, because you
> aren't using the output of the command as input to another command.
You are right.  I believe I was early on, and it just got copied and 
pasted around.
> No.  If there are two concurrent pushes occurring, the script may
> execute in parallel.
Okay, so what I'm doing is bad.  Got it.
> But you'll actually get something much worse.  receive-pack will
> create refs/for/master for the first user... and then might be put
> to sleep while another user's receive-pack starts.  That second
> user will see refs/for/master existing, and will fail their push
> because their concept of $newrev doesn't match what is currently
> at refs/for/master.  Then the first user wakes up, runs your
> post-receive, and the ref is cleared.
Yep, this is exactly what I was trying to avoid by deleting the ref.
> The better strategy would be to use an update hook that refuses to
> permit the creation of refs/for/master:
>
>    #!/bin/sh
>    ref=$1
>    old=$2
>    new=$3
>
>    case $ref in
>    refs/for/*)
>      timestamp=`date +%s`
>      git update-ref $ref-$new-$timestamp $new
>      echo "Created $ref-$new-$timestamp"
>      exit 1
>      ;;
>    *)
>      exit 0
>      ;;
>    esac
>
>
> By exit 1 here we refuse the push attempt, so receive-pack won't
> create refs/for/master, and another user pushing won't see that
> false failure.  However, unlike with Gerrit, every user is now going
> to receive a push failure message because the hook has appeared to
> reject the value, even though it accepted it.
Okay, I'll try this.  I can train people in the push failure, if 
necessary.  Thanks!

>> Before I go too much deeper down this path, am I way off base here?
> I'd have to ask why you are using gitolite and trying to abuse
> git-receive-pack to do something that Gerrit does out of the box...
>
Oh, how I would love to just leave it to Gerrit to handle this.  (I 
*really* like Gerrit.)  If you would like to just skip ahead, the actual 
question related to this email message is in the final paragraph.

We don't want to squash our topic branches before pushing them to 
Gerrit.  That means we end up with a range of 'n' number of commits in 
the Gerrit review screen.  Gerrit understands that commit 'n' is 
dependent on 'n-1' and so on, but unfortunately, it doesn't show them in 
a tree/group on the main page.  That's a minor gripe, but here is the 
major issue.

When I review Change 1, I can press a submit button, and Change 1 goes 
live right then.  Unfortunately, changes 2 through n were left behind.

If I review Change 1 through n and then press the Submit button on 
Change n and then on Change n-1 and then on Change n-2, Gerrit does the 
'right' thing (for us) and makes live the entire dependency chain at the 
same time.  Of course, I would prefer to just have a group Submit 
button, but that's another story.

This has come up on the mailing list a few times, and I even think there 
is an issue tracker item for it.

Let's ignore the review+submit portion of Gerrit now.  Can Gerrit be 
coaxed to take a refs/for/master and directly apply it to the master 
branch WITHOUT the review cycle?  If so, then I'm wasting my time trying 
to right a script.

Josh

Re: Deleting of the specified ref during the post-receive hook

[Shawn O. Pearce]

Joshua Jensen <jjensen@workspacewhiz.com> wrote:
> Let's ignore the review+submit portion of Gerrit now.  Can Gerrit be  
> coaxed to take a refs/for/master and directly apply it to the master  
> branch WITHOUT the review cycle?  If so, then I'm wasting my time trying  
> to right a script.

Nope.  But only because nobody has proposed doing that before.
Its not impossible to do...

-- 
Shawn.