[PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Jim Meyering]

This change adds a new, required, config variable: hooks.envelopesender,
and use that with sendmail's -f option.  This is important in order
to avoid relying on sendmail's "guess" at an appropriate envelope
sender address.  Without this, and in the presence of strict servers,
it is far too easy not ever to be notified, or (more insidious) never
to receive bounce email.

Below, I've included an additional patch in case you want
to require the envelopesender setting only when at least one of the
mailing list variables is set.

Signed-off-by: Jim Meyering <jim@meyering.net>
---
 templates/hooks--update |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

diff --git a/templates/hooks--update b/templates/hooks--update
index 8f6c4fe..bc1f1a9 100644
--- a/templates/hooks--update
+++ b/templates/hooks--update
@@ -12,6 +12,10 @@
 #   This is the list that all pushes will go to; leave it blank to not send
 #   emails frequently.  The log email will list every log entry in full between
 #   the old ref value and the new ref value.
+# hooks.envelopesender
+#   Use this email address as the envelope sender (argument to sendmail's
+#   -f option).  This is not the "From:" address.  This should be a useful
+#   email address within your organization.  Required.
 # hooks.announcelist
 #   This is the list that all pushes of annotated tags will go to.  Leave it
 #   blank to just use the mailinglist field.  The announce emails list the
@@ -52,10 +56,16 @@ fi
 
 # --- Config
 projectdesc=$(cat $GIT_DIR/description)
+envelope_sender=$(git-repo-config hooks.envelopesender)
 recipients=$(git-repo-config hooks.mailinglist)
 announcerecipients=$(git-repo-config hooks.announcelist)
 allowunannotated=$(git-repo-config --bool hooks.allowunannotated)
 
+if [ -z "$envelope_sender" ]; then
+	echo "Usage: hooks.envelopesender must be set in config" >&2
+	exit 1
+fi
+
 # --- Check types
 newrev_type=$(git-cat-file -t $newrev)
 
@@ -279,7 +289,7 @@ $0 $1 \\
   $3
 EOF
 #) | cat >&2
-) | /usr/sbin/sendmail -t
+) | /usr/sbin/sendmail -f "$envelope_sender" -t
 
 # --- Finished
 exit 0
-- 
1.5.1.rc1


>From 4b3d42890b2b33b2f34902d26561b78e5e92ddba Mon Sep 17 00:00:00 2001
From: Jim Meyering <jim@meyering.net>
Date: Fri, 23 Mar 2007 13:06:00 +0100
Subject: [PATCH] Fail upon undefined envelopesender only if at least one of
the mailing list variables is set.

Signed-off-by: Jim Meyering <jim@meyering.net>
---
 templates/hooks--update |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/templates/hooks--update b/templates/hooks--update
index bc1f1a9..c79b4b4 100644
--- a/templates/hooks--update
+++ b/templates/hooks--update
@@ -61,9 +61,11 @@ recipients=$(git-repo-config hooks.mailinglist)
 announcerecipients=$(git-repo-config hooks.announcelist)
 allowunannotated=$(git-repo-config --bool hooks.allowunannotated)
 
-if [ -z "$envelope_sender" ]; then
-	echo "Usage: hooks.envelopesender must be set in config" >&2
-	exit 1
+if [ -n "$recipients$announcerecipients" ]; then
+	if [ -z "$envelope_sender" ]; then
+		echo "Usage: hooks.envelopesender must be set in config" >&2
+		exit 1
+	fi
 fi
 
 # --- Check types
-- 
1.5.1.rc1

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Andy Parkins]

On Friday 2007 March 23 12:11, Jim Meyering wrote:
> This change adds a new, required, config variable: hooks.envelopesender,
> and use that with sendmail's -f option.  This is important in order
> to avoid relying on sendmail's "guess" at an appropriate envelope
> sender address.  Without this, and in the presence of strict servers,
> it is far too easy not ever to be notified, or (more insidious) never
> to receive bounce email.

Won't work.

>From "man sendmail" (although my sendmail is actually exim)

"Set  the address of the sender of a locally-generated message. This option 
can normally be used only by root or the Exim user or by one of the con-              
figured trusted users. However, anyone may use it when testing a filter file 
with -bf or when testing or verifying addresses using the -bt  or  -bv
options.  In  other  cases, the sender of a local message is always set up as 
the user who ran the exim command, and -f is ignored, with one exception."

The hook scripts run under the identity of the user doing the push; so "-f" 
won't have an effect.

I'm not sure why you would even need it; as the above quote says, the sender 
is set up as the user who ran the command.

I've only tested this with exim; perhaps it's different for other mailers.



Andy

-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Jim Meyering]

Andy Parkins <andyparkins@gmail.com> wrote:
> On Friday 2007 March 23 12:11, Jim Meyering wrote:
>> This change adds a new, required, config variable: hooks.envelopesender,
>> and use that with sendmail's -f option.  This is important in order
>> to avoid relying on sendmail's "guess" at an appropriate envelope
>> sender address.  Without this, and in the presence of strict servers,
>> it is far too easy not ever to be notified, or (more insidious) never
>> to receive bounce email.
>
> Won't work.

Thanks for the quick feedback.

It works for me using the sendmail from sendmail, and I have tested
this with the one from postfix, too.

>>From "man sendmail" (although my sendmail is actually exim)
...
> The hook scripts run under the identity of the user doing the push; so "-f"
> won't have an effect.

It has a dramatic effect for me.
With it, mail is sent, without, it's not.

> I'm not sure why you would even need it; as the above quote says, the sender
> is set up as the user who ran the command.

Part of the problem was that sendmail used a mangled hostname.
Without -f, it would use user@foo.domain.com.domain.com.

But IMHO, relying on the current behavior (using envelope sender same as
the From: address) is not an option.  It must be configurable.  I want
the envelope sender to be the same admin address for all outgoing mail,
since that's where reports of delivery problems are sent.  Besides, the
users in question don't even have "real" shell or email access on the server
system, so receiving systems would not be able to authenticate them.

In fact, some of the "users" in question are fake accounts used solely
for write access (via git-server) to the shared git repository, so the
user name exists only on the server system.

If we can't do this portably via the sendmail program, then
perhaps it's a good time to switch to using a module like Net::SMTP.

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Jakub Narebski]

Jim Meyering wrote:

> If we can't do this portably via the sendmail program, then
> perhaps it's a good time to switch to using a module like Net::SMTP.

Perhaps taking some code from contrib/continuous/
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Andy Parkins]

On Friday 2007 March 23 13:29, Jim Meyering wrote:

> It works for me using the sendmail from sendmail, and I have tested
> this with the one from postfix, too.

Is your user set as a trusted user for sendmail though?  As a normal user 
wouldn't be allowed to do it.

> It has a dramatic effect for me.
> With it, mail is sent, without, it's not.

Of course - I didn't doubt that it would, otherwise you wouldn't have bothered 
posting your patch :-)  The point is that I don't think it's universally 
applicable.  It's working for you; in your particular situation, but I'm 
guessing that you are set up as a trusted user for sendmail on your system, 
and so it's letting you rewrite the sender address.

> Part of the problem was that sendmail used a mangled hostname.
> Without -f, it would use user@foo.domain.com.domain.com.

Ah right; the fault lies not with the hook script but with your mail set up.  
I don't know how it would be done with sendmail; but for me, with exim I 
write an /etc/email-addresses file which contains:

andyp: andyparkins@gmail.com

This catches my username "andyp" on outgoing emails and rewrites the address 
to my real email "andyparkins@gmail.com".  I guess there will be an 
equivalent for other mailers.

> But IMHO, relying on the current behavior (using envelope sender same as
> the From: address) is not an option.  It must be configurable.  I want

That's not the current behaviour.  The "From:" in the email is set from the 
committer information from the revision.  The sender address is set using the 
local user account running sendmail; most email systems will ignore 
the "From:" header and let you write what you want.  The sender field, 
however is tightly controlled.

> the envelope sender to be the same admin address for all outgoing mail,
> since that's where reports of delivery problems are sent.  Besides, the

Erm: tough.  What you're asking for requires root/trusted privileges on the 
box for the user sending the email.  If it didn't, what would there be to 
stop any user sending emails that appeared to be from the admin?

> users in question don't even have "real" shell or email access on the
> server system, so receiving systems would not be able to authenticate them.

I'm not sure you've got the right idea about email; the receiving system 
doesn't have to authenticate the /user/ sending the email (if it did, then 
you'd need an account on every email server in the world).  It does need to 
authenticate the sending /machine/ (some ISPs go a bit further than that, and 
do you're sending emails that you're entitled to, but let's ignore that).  So 
whatever your email set up is; the box that has the repository must be set so 
that whomever it routes its mail to trusts it do so.  Most email servers will 
only accept incoming email for addresses local to itself; for outgoing email 
they will only relay for a defined set of machines.

> In fact, some of the "users" in question are fake accounts used solely
> for write access (via git-server) to the shared git repository, so the
> user name exists only on the server system.

They're not fake accounts then - they really exist on that system.  I'm 
guessing by "fake" you mean that their login shell is set to git-shell.  That 
doesn't stop git-server from running as their UID, and in turn running the 
hook script under their UID.  The difficulty is that those users really are 
sending email - if your system is set so that they're not allowed to then the 
act of using git shouldn't magically give them that permission.

> If we can't do this portably via the sendmail program, then
> perhaps it's a good time to switch to using a module like Net::SMTP.

I don't think that's the right solution either.  That's just making the hook 
script be a mail transfer agent itself, and so it would then connect to your 
receiving server directly.  That implies that it would have to be able to 
solve all the problems that a real MTA solves itself - I don't think it's the 
right place for that level of functionality.

So; that was a whole load of negative stuff, here's some suggestions 
(untested):
 * Run the hook script SUID.  Pretty nasty.
 * Configure /etc/passwd so the fake users all map to the same UID; then use
   that username in /etc/email-addresses (or equivalent) to set up the admin
   address you want to use.
 * Change your local mail setup to include the fake users as trusted

Does anyone know how other VCSs solve this problem?  Short of running some 
sort of privileged process I can't think of a way.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Jim Meyering]

Andy Parkins <andyparkins@gmail.com> wrote:

> On Friday 2007 March 23 13:29, Jim Meyering wrote:
>
>> It works for me using the sendmail from sendmail, and I have tested
>> this with the one from postfix, too.
>
> Is your user set as a trusted user for sendmail though?

No.  There is no need for that.

> As a normal user wouldn't be allowed to do it.

You mean with exim's sendmail?
sendmail -f alt-envelope-sender works just fine when run by a
non-privileged user when it's Sendmail's sendmail or the one from Postfix.
It's a shame if it doesn't work with exim's implementation.

>> It has a dramatic effect for me.
>> With it, mail is sent, without, it's not.
>
> Of course - I didn't doubt that it would, otherwise you wouldn't have bothered
> posting your patch :-)  The point is that I don't think it's universally
> applicable.  It's working for you; in your particular situation, but I'm
> guessing that you are set up as a trusted user for sendmail on your system,
> and so it's letting you rewrite the sender address.

There is no need to declare a user "trusted" for sendmail to allow
it to use the -f option.

> That's not the current behaviour.  The "From:" in the email is set from the
> committer information from the revision.  The sender address is set using the
> local user account running sendmail; most email systems will ignore
> the "From:" header and let you write what you want.  The sender field,
> however is tightly controlled.

Thanks for clarifying.  However, in any case, the current behavior
is too constraining.  I've needed to change it now in two totally
different environments, so I really doubt that this is an isolated
case.  FYI, the first case was to allow commit email to be sent from
the GNU coreutils git repository in a vserver on savannah.gnu.org.
The second was at work last night.

>> the envelope sender to be the same admin address for all outgoing mail,
>> since that's where reports of delivery problems are sent.  Besides, the
>
> Erm: tough.  What you're asking for requires root/trusted privileges on the
> box for the user sending the email.

No.
As I tried to make clear, above, for the sendmail programs I use,
sendmail -f works fine for unprivileged users.

> If it didn't, what would there be to
> stop any user sending emails that appeared to be from the admin?

Nothing.  We've always been able to do that, though when we do,
we also get an X-Authentication-Warning header.

>> users in question don't even have "real" shell or email access on the
>> server system, so receiving systems would not be able to authenticate them.
>
> I'm not sure you've got the right idea about email; the receiving system
> doesn't have to authenticate the /user/ sending the email (if it did, then

Not the *final* recipient, of course.
Some *internal* MX servers are very careful about sender addresses
in outbound mail, verifying them before forwarding the message on.

...
Thanks for the explanation of how email works, but you may
assume that I know a little more than the average hacker :-)

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Andy Parkins]

On Friday 2007 March 23 15:25, Jim Meyering wrote:
> Andy Parkins <andyparkins@gmail.com> wrote:
> > On Friday 2007 March 23 13:29, Jim Meyering wrote:
> >> It works for me using the sendmail from sendmail, and I have tested
> >> this with the one from postfix, too.
> >
> > Is your user set as a trusted user for sendmail though?
>
> No.  There is no need for that.
>
> > As a normal user wouldn't be allowed to do it.
>
> You mean with exim's sendmail?
> sendmail -f alt-envelope-sender works just fine when run by a
> non-privileged user when it's Sendmail's sendmail or the one from Postfix.
> It's a shame if it doesn't work with exim's implementation.

Okay; I went and found a sendmail manpage:

"-fname       Sets the name of the ``from'' person (i.e., the sender of the
              mail).  -f can only be used by ``trusted'' users (normally
              root,  daemon, and network) or if the person you are trying to
              become is the same as the person you are."

Seems the same as exim to me.  Perhaps your distribution sets it up to allow 
if for anyone?

> case.  FYI, the first case was to allow commit email to be sent from
> the GNU coreutils git repository in a vserver on savannah.gnu.org.
> The second was at work last night.

> Not the *final* recipient, of course.
> Some *internal* MX servers are very careful about sender addresses
> in outbound mail, verifying them before forwarding the message on.

Fair enough.

> Thanks for the explanation of how email works, but you may
> assume that I know a little more than the average hacker :-)

My apologies; I didn't mean to be patronising - I wrongly assumed you were 
just confused about who was authenticating whom.

So; in short: I don't have a good answer to offer you, and it seems that "-f" 
is working for you.  However, I'm still not convinced that this is the 
correct thing to do in the default hook.  My main gripe is still all 
these "trusted user" paragraphs in the MTA manual pages.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Jim Meyering]

> On Friday 2007 March 23 15:25, Jim Meyering wrote:
>> Andy Parkins <andyparkins@gmail.com> wrote:
>> > On Friday 2007 March 23 13:29, Jim Meyering wrote:
>> >> It works for me using the sendmail from sendmail, and I have tested
>> >> this with the one from postfix, too.
>> >
>> > Is your user set as a trusted user for sendmail though?
>>
>> No.  There is no need for that.
>>
>> > As a normal user wouldn't be allowed to do it.
>>
>> You mean with exim's sendmail?
>> sendmail -f alt-envelope-sender works just fine when run by a
>> non-privileged user when it's Sendmail's sendmail or the one from Postfix.
>> It's a shame if it doesn't work with exim's implementation.
>
> Okay; I went and found a sendmail manpage:
>
> "-fname       Sets the name of the ``from'' person (i.e., the sender of the
>               mail).  -f can only be used by ``trusted'' users (normally
>               root,  daemon, and network) or if the person you are trying to
>               become is the same as the person you are."
>
> Seems the same as exim to me.  Perhaps your distribution sets it up to allow
> if for anyone?

I suspect that the above is from older documentation.
As you'll see in the excerpts below, the "can only be used"
is replaced with "should...", to permit using -f in cases like mine.

The documentation from sendmail-8.11.7/doc/op/op.me says this:

-f addr   The envelope sender address is set to addr.  This
          address  may  also  be used in the From: header if
          that header is missing during initial  submission.
          The envelope sender address is used as the recipi-
          ent for delivery status notifications and may also
          appear in a Return-Path: header.

Then, in cf/README, they explain the trusted-users file is solely
to avoid a warning:

use_ct_file     Read the file /etc/mail/trusted-users file to get the
                names of users that will be ``trusted'', that is, able to
                set their envelope from address using -f without generating
                a warning message.  The actual filename can be overridden
                by redefining confCT_FILE.

A quick search found lots like this:
  http://www.linuxmanpages.com/man8/sendmail.8.php

    -fname
        Sets the name of the ``from'' person (i.e., the envelope sender of
        the mail). This address may also be used in the From: header if
        that header is missing during initial submission. The envelope
        sender address is used as the recipient for delivery status
        notifications and may also appear in a Return-Path: header. -f
        should only be used by ``trusted'' users (normally root, daemon,
        and network) or if the person you are trying to become is the
        same as the person you are. Otherwise, an X-Authentication-Warning
        header will be added to the message.

So that was classic sendmail (8.11.7).  Here's the description of -f
for postfix's sendmail (man sendmail):

       -f sender
              Set the envelope sender  address.  This  is  the  address  where
              delivery problems are sent to. With Postfix versions before 2.1,
              the  Errors-To:  message  header  overrides  the  error   return
              address.

In addition, I have tested this by sending myself a message
via printf '...' |sendmail -f nobody@nowhere.com -oi -t -v, and
examined the headers in the received messages.  The first I sent
from a system running sendmail-8.11.7, the second from one running
Debian/unstable's Postfix 2.3.8-2.  Both were sent by a "regular" (non-root)
user, and the envelope sender was the requested "nobody@nowhere.com"
in each case.

> So; in short: I don't have a good answer to offer you, and it seems that "-f"
> is working for you.

Yes, it does work for me.  And it would work for anyone with
sendmail or Postfix.  Whether it works for an exim-based
sendmail is a question of policy, and the default in Debian-based
systems is to allow it:

>From /etc/exim4/conf.d/main/02_exim4-config_options:

  .ifndef MAIN_FORCE_SENDER
  local_from_check = false
  local_sender_retain = true
  untrusted_set_sender = *
  .endif

> However, I'm still not convinced that this is the
> correct thing to do in the default hook.  My main gripe is still all
> these "trusted user" paragraphs in the MTA manual pages.

There is a legitimate need for this functionality, and -f does
usually work, so how about a compromise:

  Include support for using sendmail's "-f envelope-sender" option
  in the default hook, but enable it only if/when hooks.envelopesender
  is set in the config file.

Re: [PATCH] hooks--update: new, required, config variable: hooks.envelopesender,

[Andy Parkins]

On Saturday 2007, March 24, Jim Meyering wrote:

> There is a legitimate need for this functionality, and -f does
> usually work, so how about a compromise:
>
>   Include support for using sendmail's "-f envelope-sender" option
>   in the default hook, but enable it only if/when
> hooks.envelopesender is set in the config file.

Perfect.  Now that I can't think of any objection to.  


Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

[PATCH] hooks--update: new, optional, config variable: hooks.envelopesender

[Jim Meyering]

This change adds a new, optional, config variable: hooks.envelopesender.
When set, the "update" hook uses its value with sendmail's -f option.

Signed-off-by: Jim Meyering <jim@meyering.net>
---
 templates/hooks--update |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

diff --git a/templates/hooks--update b/templates/hooks--update
index 1a60773..02e2494 100644
--- a/templates/hooks--update
+++ b/templates/hooks--update
@@ -12,6 +12,11 @@
 #   This is the list that all pushes will go to; leave it blank to not send
 #   emails frequently.  The log email will list every log entry in full between
 #   the old ref value and the new ref value.
+# hooks.envelopesender
+#   Use this email address as the envelope sender (argument to sendmail's
+#   -f option).  This is not the "From:" address.  This can be useful to
+#   ensure that all delivery problem notifications go to the same, known-
+#   working address.
 # hooks.announcelist
 #   This is the list that all pushes of annotated tags will go to.  Leave it
 #   blank to just use the mailinglist field.  The announce emails list the
@@ -52,10 +57,15 @@ fi

 # --- Config
 projectdesc=$(cat $GIT_DIR/description)
+envelope_sender=$(git-repo-config hooks.envelopesender)
 recipients=$(git-repo-config hooks.mailinglist)
 announcerecipients=$(git-repo-config hooks.announcelist)
 allowunannotated=$(git-repo-config --bool hooks.allowunannotated)

+if [ -n "$envelope_sender" ]; then
+	envelope_sender_option="-f '$envelope_sender'"
+fi
+
 # check for no description
 if [ -z "$projectdesc" -o "$projectdesc" = "Unnamed repository; edit this file to name it for gitweb" ]; then
 	echo "*** Project description file hasn't been set" >&2
@@ -285,7 +295,7 @@ $0 $1 \\
   $3
 EOF
 #) | cat >&2
-) | /usr/sbin/sendmail -t
+) | eval /usr/sbin/sendmail $envelope_sender_option -t

 # --- Finished
 exit 0