[PATCH] post-receive-email: do not call sendmail if no mail was generated

[PATCH] post-receive-email: do not call sendmail if no mail was generated

[Lars Noschinski]

contrib/hooks/post-receive-email used to call the send_mail function
(and thus, /usr/sbin/sendmail), even if generate_mail returned an error.
This is problematic, as the sendmail binary provided by exim4 generates
an error mail if provided with an empty input.

Therefore, this commit changes post-receive-email to only call sendmail
if generate_mail returned without error.

Signed-off-by: Lars Noschinski <lars@public.noschinski.de>
---
 contrib/hooks/post-receive-email |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

Obvious drawback of this solution is that the mails are kept in memory as a
whole. But the mails should be small enough, so that this does not hurt.

I'm not sure how to write a test for this bug, as the hook (correctly) uses an
absolute path to call sendmail.

diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email
index 2a66063..818a270 100755
--- a/contrib/hooks/post-receive-email
+++ b/contrib/hooks/post-receive-email
@@ -684,6 +684,9 @@ if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
 else
 	while read oldrev newrev refname
 	do
-		generate_email $oldrev $newrev $refname | send_mail
+		mail="$(generate_email $oldrev $newrev $refname)"
+		if [ $? -eq 0 ]; then
+			printf '%s' "$mail" | send_mail
+		fi
 	done
 fi
-- 
1.6.3.3

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Lars Noschinski]

* Lars Noschinski <lars@public.noschinski.de> [09-08-28 19:39]:
> contrib/hooks/post-receive-email used to call the send_mail function
> (and thus, /usr/sbin/sendmail), even if generate_mail returned an error.
> This is problematic, as the sendmail binary provided by exim4 generates
> an error mail if provided with an empty input.
> 
> Therefore, this commit changes post-receive-email to only call sendmail
> if generate_mail returned without error.
> 
> Signed-off-by: Lars Noschinski <lars@public.noschinski.de>

Is anything wrong with this patch? Or is it just queued to be committed
some time?

 - Lars.

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Junio C Hamano]

Lars Noschinski <lars@public.noschinski.de> writes:

> * Lars Noschinski <lars@public.noschinski.de> [09-08-28 19:39]:
>> contrib/hooks/post-receive-email used to call the send_mail function
>> (and thus, /usr/sbin/sendmail), even if generate_mail returned an error.
>> This is problematic, as the sendmail binary provided by exim4 generates
>> an error mail if provided with an empty input.
>> 
>> Therefore, this commit changes post-receive-email to only call sendmail
>> if generate_mail returned without error.
>> 
>> Signed-off-by: Lars Noschinski <lars@public.noschinski.de>
>
> Is anything wrong with this patch? Or is it just queued to be committed
> some time?

It is not queued anywhere as far as I am concerned.

I was waiting for others to review the patch and nothing happened, so the
patch was in limbo.  Thanks for sending a reminder message I am responding
to.  You did the right thing when nothing happened to a patch that did not
see any discussion.

You can avoid this by CC'ing people who have been involved in the past
with the parts of the system you are patching in the initial posting of
your patch (I am not one of them, so CC'ing me didn't help).

Here are my knee-jerk reactions to the patch after a quick glance, without
thinking deeply nor looking at the other parts of the file you did not
touch, but looking only at the parts shown in your patch:

 - Slurping generate_email's output into a shell variable is a bad taste.
   You said that the message is always small enough but _how_ do we know
   it?

 - If this is to save us from a quirk in some but not all implementations
   of /usr/lib/sendmail, then shouldn't the logic be made into a new
   conditional?

 - I do not see a direct link between "if generate_mail returned an error"
   and "if ... an empty input".  What if generate_mail started its output
   but then failed halfway?  We have some output so the send_mail won't be
   fed empty, but $? would be not zero, so the patch is testing a
   different condition from what the log message claims to be checking.

People who do use this script and people who have worked on it may have
other more useful comments.

Thanks.

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Lars Noschinski]

* Junio C Hamano <gitster@pobox.com> [09-09-08 19:22]:
> Lars Noschinski <lars@public.noschinski.de> writes:
> > * Lars Noschinski <lars@public.noschinski.de> [09-08-28 19:39]:
> >> contrib/hooks/post-receive-email used to call the send_mail function
> >> (and thus, /usr/sbin/sendmail), even if generate_mail returned an error.
> >> This is problematic, as the sendmail binary provided by exim4 generates
> >> an error mail if provided with an empty input.
> >> 
> >> Therefore, this commit changes post-receive-email to only call sendmail
> >> if generate_mail returned without error.
> >> 
> >> Signed-off-by: Lars Noschinski <lars@public.noschinski.de>
[...]
>  - Slurping generate_email's output into a shell variable is a bad taste.
>    You said that the message is always small enough but _how_ do we know
>    it?

You are right; I overlooked that the revision formatting is configurable
and if set up to display the full patch, the mail could get pretty big.

I know found a solution which does neither store the full output in a
variable nor needs a temporary file. I will post it as a reply to this
mail.

>  - If this is to save us from a quirk in some but not all implementations
>    of /usr/lib/sendmail, then shouldn't the logic be made into a new
>    conditional?

I don't know if this a quirk in exim; I could not find a formal
specification of the sendmail behaviour and treating such an "input" as
an error seems at least not insane.

In any case, I think the overhead implied by new patch is small enough,
that a switch is unnecessary.

>  - I do not see a direct link between "if generate_mail returned an error"
>    and "if ... an empty input".  What if generate_mail started its output
>    but then failed halfway?  We have some output so the send_mail won't be
>    fed empty, but $? would be not zero, so the patch is testing a
>    different condition from what the log message claims to be checking.

Yeah, you are right. This is also fixed in the new patch.

> People who do use this script and people who have worked on it may have
> other more useful comments.

CCed some of them.

[PATCH] post-receive-email: do not call sendmail if no mail was generated

[Lars Noschinski]

contrib/hooks/post-receive-email used to call the send_mail function
(and thus, /usr/sbin/sendmail), even if generate_mail generated no
output.  This is problematic, as the sendmail binary provided by exim4
generates an error mail if provided with an empty input.

Therefore, we now read one line ourselves and use the result to decide
if we really want to call /usr/sbin/sendmail.
---
 contrib/hooks/post-receive-email |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

Two things changed:

 - we do not read the whole mail in a shell variable
 - the decision whether to call sendmail is based on the output generated
   by generate_mail, not its return code

diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email
index 2a66063..c855c31 100755
--- a/contrib/hooks/post-receive-email
+++ b/contrib/hooks/post-receive-email
@@ -637,6 +637,16 @@ show_new_revisions()
 
 send_mail()
 {
+	OIFS=$IFS
+	IFS='
+'
+	read FIRSTLINE || exit 1
+	(printf $FIRSTLINE'\n'; cat) | call_sendmail
+	IFS=$OLD_IFS
+}
+
+call_sendmail()
+{
 	if [ -n "$envelopesender" ]; then
 		/usr/sbin/sendmail -t -f "$envelopesender"
 	else
@@ -644,6 +654,7 @@ send_mail()
 	fi
 }
 
+
 # ---------------------------- main()
 
 # --- Constants
-- 
1.6.3.3

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Junio C Hamano]

Lars Noschinski <lars@public.noschinski.de> writes:

> contrib/hooks/post-receive-email used to call the send_mail function
> (and thus, /usr/sbin/sendmail), even if generate_mail generated no
> output.  This is problematic, as the sendmail binary provided by exim4
> generates an error mail if provided with an empty input.

I actually have a bigger question, not about the implementation but about
the cause.

If generate_email results in an empty output in this codepath:

	# Check if we've got anyone to send to
	if [ -z "$recipients" ]; then
		...
		echo >&2 "*** $config_name is not set so no email will be sent"
		echo >&2 "*** for $refname update $oldrev->$newrev"
		exit 0
	fi

shouldn't we rather receive an error e-mail than let the
misconfiguration go undetected?

Before this check, I do not see anywhere generate_email would return nor
exit, and after this check, there is a call to generate_email_header and
that guarantees that the output from the generate_email function is not
empty, so it looks to me that triggering this check is the only case your
patch would change the behaviour of the script.

It looks to me that your exim error mail is actually reporting a
legitimate problem you would want to fix in your configuration.

> Therefore, we now read one line ourselves and use the result to decide
> if we really want to call /usr/sbin/sendmail.
> ---
>  contrib/hooks/post-receive-email |   11 +++++++++++
>  1 files changed, 11 insertions(+), 0 deletions(-)
>
> Two things changed:
>
>  - we do not read the whole mail in a shell variable
>  - the decision whether to call sendmail is based on the output generated
>    by generate_mail, not its return code
>
> diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email
> index 2a66063..c855c31 100755
> --- a/contrib/hooks/post-receive-email
> +++ b/contrib/hooks/post-receive-email
> @@ -637,6 +637,16 @@ show_new_revisions()
>  
>  send_mail()
>  {
> +	OIFS=$IFS
> +	IFS='
> +'
> +	read FIRSTLINE || exit 1

Shouldn't this be merely a "return"?  The caller looks like this:

	while read oldrev newrev refname
	do
		generate_email $oldrev $newrev $refname | send_mail
	done

and you would not want to stop after punting to report on the first ref.

> +	(printf $FIRSTLINE'\n'; cat) | call_sendmail
> +	IFS=$OLD_IFS
> +}
> +
> +call_sendmail()
> +{
>  	if [ -n "$envelopesender" ]; then
>  		/usr/sbin/sendmail -t -f "$envelopesender"
>  	else
> @@ -644,6 +654,7 @@ send_mail()
>  	fi
>  }
>  
> +
>  # ---------------------------- main()
>  
>  # --- Constants

Why?

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Lars Noschinski]

* Junio C Hamano <gitster@pobox.com> [09-09-08 22:15]:
> Lars Noschinski <lars@public.noschinski.de> writes:
> 
> > contrib/hooks/post-receive-email used to call the send_mail function
> > (and thus, /usr/sbin/sendmail), even if generate_mail generated no
> > output.  This is problematic, as the sendmail binary provided by exim4
> > generates an error mail if provided with an empty input.
> 
> I actually have a bigger question, not about the implementation but about
> the cause.
> 
> If generate_email results in an empty output in this codepath:
> 
> 	# Check if we've got anyone to send to
> 	if [ -z "$recipients" ]; then
> 		...
> 		echo >&2 "*** $config_name is not set so no email will be sent"
> 		echo >&2 "*** for $refname update $oldrev->$newrev"
> 		exit 0
> 	fi
> 
> shouldn't we rather receive an error e-mail than let the
> misconfiguration go undetected?

Probably not. The error message is displayed to the user who did the
push. Normally (if no explicit From: address is configured), this is the
same user, which would receive the error mail.

> Before this check, I do not see anywhere generate_email would return nor
> exit, and after this check, there is a call to generate_email_header and
> that guarantees that the output from the generate_email function is not
> empty, so it looks to me that triggering this check is the only case your
> patch would change the behaviour of the script.

Actually, there are a two cases in the case statement before, where
generate_email would return:

    refs/remotes/*,commit)
        # tracking branch
        refname_type="tracking branch"
        short_refname=${refname##refs/remotes/}
        echo >&2 "*** Push-update of tracking branch, $refname"
        echo >&2 "***  - no email generated."
        exit 0
        ;;
    *)
        # Anything else (is there anything else?)
        echo >&2 "*** Unknown type of update to $refname ($rev_type)"
        echo >&2 "***  - no email generated"
        exit 1
        ;;

i.e. if we are pushing to a branch neither in refs/tags nor refs/heads.

In our setting, the build process pushes to refs/builds, so we can track
code changes between different builds, without displaying a whole lot of
mostly useless branches or tags to the user.

> > diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email
> > index 2a66063..c855c31 100755
> > --- a/contrib/hooks/post-receive-email
> > +++ b/contrib/hooks/post-receive-email
> > @@ -637,6 +637,16 @@ show_new_revisions()
> >  
> >  send_mail()
> >  {
> > +	OIFS=$IFS
> > +	IFS='
> > +'
> > +	read FIRSTLINE || exit 1
> 
> Shouldn't this be merely a "return"?  The caller looks like this:

Yes.

I'll fix it in the next patch (when there are further comments); but you
may fold it in (and add the SOB if forgot), if you prefer.

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Junio C Hamano]

Lars Noschinski <lars-2008-2@usenet.noschinski.de> writes:

> Actually, there are a two cases in the case statement before, where
> generate_email would return:
>
>     refs/remotes/*,commit)
>         # tracking branch
>         refname_type="tracking branch"
>         short_refname=${refname##refs/remotes/}
>         echo >&2 "*** Push-update of tracking branch, $refname"
>         echo >&2 "***  - no email generated."
>         exit 0
>         ;;
>     *)
>         # Anything else (is there anything else?)
>         echo >&2 "*** Unknown type of update to $refname ($rev_type)"
>         echo >&2 "***  - no email generated"
>         exit 1
>         ;;

Ok, that justifies the existence of the patch.

Re: [PATCH] post-receive-email: do not call sendmail if no mail was generated

[Andy Parkins]

Thanks for CCing me in - I don't monitor the list closely enough these days 
:-)

Junio C Hamano wrote:

> If generate_email results in an empty output in this codepath:
> 
> # Check if we've got anyone to send to
> if [ -z "$recipients" ]; then
> ...
> echo >&2 "*** $config_name is not set so no email will be sent"
> echo >&2 "*** for $refname update $oldrev->$newrev"
> exit 0
> fi
> 
> shouldn't we rather receive an error e-mail than let the
> misconfiguration go undetected?

I don't know if it's still the case, but when I wrote it, anything that went 
to standard error appeared on the client terminal, however, it could 
probably do with being a better description of who is generating the 
message, otherwise it'll be some anonymous error during a push, giving the 
user no clue as to how to fix it.

> Before this check, I do not see anywhere generate_email would return nor
> exit, and after this check, there is a call to generate_email_header and
> that guarantees that the output from the generate_email function is not
> empty, so it looks to me that triggering this check is the only case your
> patch would change the behaviour of the script.

There is also a check for the validity of the update type above the 
recipients check.

I'm wondering actually if all of these should be "return"s rather than 
"exit"s.  Better still would be if there were some sort of exception 
throwing mechanism in shell script - anyone know if there is?



Andy

P.S. Hope you're all keeping well.

-- 
Dr Andy Parkins
andyparkins@gmail.com