git-send-email: Send with mutt(1)

git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

Hi!

I'm trying to use mutt(1) with git-send-email(1).  Is that possible?
I tried --sendmail-cmd=mutt, but am getting strange errors.
The reason I want to send with mutt(1) is because it can encrypt and
sign mail (with some tweaks), which git-send-mail(1) doesn't.

Thanks,
Alex
-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Jeff King]

On Tue, Nov 07, 2023 at 12:14:21PM +0100, Alejandro Colomar wrote:

> I'm trying to use mutt(1) with git-send-email(1).  Is that possible?
> I tried --sendmail-cmd=mutt, but am getting strange errors.
> The reason I want to send with mutt(1) is because it can encrypt and
> sign mail (with some tweaks), which git-send-mail(1) doesn't.

I think there's a lot of overlap between what git-send-email does and
what mutt does, to the point that you probably don't need to use
send-email at all.

I assume what you want out of send-email here is the actual generation
of patch emails. But under the hood that is all done by git-format-patch
anyway. So for example if you do:

  git format-patch --stdout origin..HEAD >patches
  mutt -f patches

And then you can use mutt's "resend-message" function to send each one.
I use config like this:

  macro index,pager b ":set edit_headers=yes<enter><resend-message>:set edit_headers=no<enter>"

If you're sending a long series, it's helpful to pre-populate various
headers in the format-patch command with "--to", etc. I usually do so by
sending the cover letter directly via mutt, and then using some perl
hackery to convert those headers into format-patch args. The script I
use is below (it will also, when run without a terminal, generate the
patch summary for the cover letter; I use it with "r!my-script" while
writing the cover letter in vim).

(This script is what I use every day, so it should be fairly robust. But
it is also over 15 years old, so I don't promise there isn't a simpler
way to do some of what it does ;) ).

-- >8 --
#!/bin/sh
upstream_branch() {
  current=`git symbolic-ref HEAD`
  upstream=`git for-each-ref --format='%(upstream)' "$current"`
  if test -n "$upstream"; then
    echo $upstream
  else
    echo origin
  fi
}

get_reply_headers() {
  perl -ne '
    if (defined $opt) {
      if (/^\s+(.*)/) {
        $val .= " $1";
        next;
      }
      print "--$opt=", quotemeta($val), " ";
      $opt = $val = undef;
    }
    if (/^(cc|to):\s*(.*)/i) {
      $opt = lc($1);
      $val = $2;
    }
    elsif (/^message-id:\s*(.*)/i) {
      $opt = "in-reply-to";
      $val = $1;
    }
    elsif (/^subject:\s*\[PATCH v(\d+)/i) {
      print "-v$1 ";
    }
    elsif (/^$/) {
      last;
    }
  '
}

format_patch() {
  git format-patch -s --stdout --from "$@"
}

has_nonoption=
for i in "$@"; do
  case "$i" in
    -[0-9]*) has_nonoption=yes ;;
    -*) ;;
     *) has_nonoption=yes
  esac
done

: ${REPLY:=$HOME/patch}
test -e "$REPLY" && eval "set -- `get_reply_headers <\"$REPLY\"` \"\$@\""
test "$has_nonoption" = "yes" || set -- "$@" `upstream_branch`

if test -t 1; then
  format_patch "$@" >.mbox
  mutt -e 'set sort=mailbox-order' -f .mbox
  rm -f .mbox
else
  format_patch "$@" |
  perl -lne '
    if (/^Subject: (.*)/) {
      $subject = $1;
    }
    elsif ($subject && /^\s+(.*)/) {
      $subject .= " $1";
    }
    elsif ($subject) {
      print $subject;
      $subject = undef;
    }
  ' |
  sed -e 's/\[PATCH /[/' \
      -e 's/]/]:/' \
      -e 's/^/  /'
  echo
  format_patch --cover-letter "$@" |
  sed -ne '/|/,/^$/p; /^-- /q'
fi

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3961 bytes --]

Hi Jeff,

On Tue, Nov 07, 2023 at 12:48:03PM -0500, Jeff King wrote:
> I think there's a lot of overlap between what git-send-email does and
> what mutt does, to the point that you probably don't need to use
> send-email at all.
> 
> I assume what you want out of send-email here is the actual generation
> of patch emails. But under the hood that is all done by git-format-patch
> anyway. So for example if you do:

Yeah, most of it is done by format-patch.  There are few things I
actually need from send-email.  One of them is generating the Cc from
the sign-offs and other tags found in the patch.

I had been thinking these days that it would be useful to have
format-patch generate those.  How about adding a --signed-off-by-cc to
format-patch?

> 
>   git format-patch --stdout origin..HEAD >patches
>   mutt -f patches
> 
> And then you can use mutt's "resend-message" function to send each one.
> I use config like this:
> 
>   macro index,pager b ":set edit_headers=yes<enter><resend-message>:set edit_headers=no<enter>"
> 
> If you're sending a long series, it's helpful to pre-populate various
> headers in the format-patch command with "--to", etc. I usually do so by
> sending the cover letter directly via mutt, and then using some perl
> hackery to convert those headers into format-patch args. The script I

Indeed, that hackery is what send-email already does, so how about
moving those features a bit upstream so that format-patch can do them
too?

Although then, maybe it's simpler to teach send-email to learn to use
mutt(1) under the hood for the actual send.

> use is below (it will also, when run without a terminal, generate the
> patch summary for the cover letter; I use it with "r!my-script" while
> writing the cover letter in vim).
> 
> (This script is what I use every day, so it should be fairly robust. But
> it is also over 15 years old, so I don't promise there isn't a simpler
> way to do some of what it does ;) ).
> 
> -- >8 --
> #!/bin/sh
> upstream_branch() {
>   current=`git symbolic-ref HEAD`
>   upstream=`git for-each-ref --format='%(upstream)' "$current"`
>   if test -n "$upstream"; then
>     echo $upstream
>   else
>     echo origin
>   fi
> }
> 
> get_reply_headers() {
>   perl -ne '
>     if (defined $opt) {
>       if (/^\s+(.*)/) {
>         $val .= " $1";
>         next;
>       }
>       print "--$opt=", quotemeta($val), " ";
>       $opt = $val = undef;
>     }
>     if (/^(cc|to):\s*(.*)/i) {
>       $opt = lc($1);
>       $val = $2;
>     }
>     elsif (/^message-id:\s*(.*)/i) {
>       $opt = "in-reply-to";
>       $val = $1;
>     }
>     elsif (/^subject:\s*\[PATCH v(\d+)/i) {
>       print "-v$1 ";
>     }
>     elsif (/^$/) {
>       last;
>     }
>   '
> }
> 
> format_patch() {
>   git format-patch -s --stdout --from "$@"
> }
> 
> has_nonoption=
> for i in "$@"; do
>   case "$i" in
>     -[0-9]*) has_nonoption=yes ;;
>     -*) ;;
>      *) has_nonoption=yes
>   esac
> done
> 
> : ${REPLY:=$HOME/patch}
> test -e "$REPLY" && eval "set -- `get_reply_headers <\"$REPLY\"` \"\$@\""
> test "$has_nonoption" = "yes" || set -- "$@" `upstream_branch`
> 
> if test -t 1; then
>   format_patch "$@" >.mbox
>   mutt -e 'set sort=mailbox-order' -f .mbox
>   rm -f .mbox
> else
>   format_patch "$@" |
>   perl -lne '
>     if (/^Subject: (.*)/) {
>       $subject = $1;
>     }
>     elsif ($subject && /^\s+(.*)/) {
>       $subject .= " $1";
>     }
>     elsif ($subject) {
>       print $subject;
>       $subject = undef;
>     }
>   ' |
>   sed -e 's/\[PATCH /[/' \
>       -e 's/]/]:/' \
>       -e 's/^/  /'
>   echo
>   format_patch --cover-letter "$@" |
>   sed -ne '/|/,/^$/p; /^-- /q'
> fi

Thanks!  I'll try it.  Although I don't know perl, so I hope I don't
need to tweak it much.  :)

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Jeff King]

On Tue, Nov 07, 2023 at 07:36:44PM +0100, Alejandro Colomar wrote:

> > I assume what you want out of send-email here is the actual generation
> > of patch emails. But under the hood that is all done by git-format-patch
> > anyway. So for example if you do:
> 
> Yeah, most of it is done by format-patch.  There are few things I
> actually need from send-email.  One of them is generating the Cc from
> the sign-offs and other tags found in the patch.
> 
> I had been thinking these days that it would be useful to have
> format-patch generate those.  How about adding a --signed-off-by-cc to
> format-patch?

That seems like a reasonable feature. Probably it should be
--cc-from-trailer=signed-off-by, and then you could do the same with
other trailers.

It feels like you could _almost_ do it with the existing
--format='%(trailers)' functionality, but there's no way to say "do the
regular --format=email output, but also stick this extra format in the
headers section". Plus there are probably some niceties you'd get from
Git knowing that you're adding headers (like de-duping addresses).

That feature might end up somewhat hairy, though, as then you get into
questions of parsing address lists, etc. We do all that now in perl with
send-email, where we can lean on some parsing libraries. So I dunno.

> > If you're sending a long series, it's helpful to pre-populate various
> > headers in the format-patch command with "--to", etc. I usually do so by
> > sending the cover letter directly via mutt, and then using some perl
> > hackery to convert those headers into format-patch args. The script I
> 
> Indeed, that hackery is what send-email already does, so how about
> moving those features a bit upstream so that format-patch can do them
> too?

Yeah, if they existed in format-patch I might be able to reuse them. I
am hesitant, though, just because handling all the corner cases on
parsing is going to be a bit of new C code.

> Although then, maybe it's simpler to teach send-email to learn to use
> mutt(1) under the hood for the actual send.

I think you will find some corner cases in trying to make mutt act just
like an mta accepting delivery. Two I can think of:

  1. It will take a body on stdin, but not a whole message. We can hack
     around that with some postponed-folder magic, though.

  2. Bcc headers are stripped before sendmail sees the message (but
     those addresses appear on the command-line). Converting that back
     to bcc so that mutt can then re-strip them would be annoying but
     possible. If you don't use bcc, it probably makes sense to just
     punt on this.

So maybe a script like this:

-- >8 --
#!/bin/sh

# ignore arguments; mutt will parse them itself
# from to/cc headers. Note that we'll miss bcc this
# way, but handling that would probably be kind of
# tricky; we'd need to re-add those recipients as actual
# bcc headers so that mutt knows how to handle them.

# spool the message to a fake mbox; we need to add
# a "From" line to make it look legit
trap 'rm -f to-send' 0 &&
{
  echo "From whatever Mon Sep 17 00:00:00 2001" &&
  cat
} >to-send &&

# and then have mutt "resume" it. We have to redirect
# stdin back from the terminal, since ours is a pipe
# with the message contents.
mutt -p \
  -e 'set postponed=to-send' \
  -e 'set edit_headers=yes' \
  </dev/tty
-- 8< --

and then in your git config:

  [sendemail]
  sendmailcmd = /path/to/mutt-as-mta.sh

There are mutt-specific bits there that I don't think send-email should
have to know about. Perhaps there are generic options that send-email
could learn, but it really feels like you'd do better teaching mutt to
be more ready to handle this (like taking a whole message on stdin,
headers and all, rather than just a body).

-Peff

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 4632 bytes --]

Hi Jeff!

On Tue, Nov 07, 2023 at 03:16:55PM -0500, Jeff King wrote:
> > I had been thinking these days that it would be useful to have
> > format-patch generate those.  How about adding a --signed-off-by-cc to
> > format-patch?
> 
> That seems like a reasonable feature. Probably it should be
> --cc-from-trailer=signed-off-by, and then you could do the same with
> other trailers.

That would work for me.  I only suggested the other one because it's
aleady in send-email in that form.  But yeah, it might be useful to have
finer control.  In my case, something that CCs every mail in the trailer
would work (Reviewed-by, Suggested-by, ...  I want them all, always).

> 
> It feels like you could _almost_ do it with the existing
> --format='%(trailers)' functionality, but there's no way to say "do the
> regular --format=email output, but also stick this extra format in the
> headers section". Plus there are probably some niceties you'd get from
> Git knowing that you're adding headers (like de-duping addresses).
> 
> That feature might end up somewhat hairy, though, as then you get into
> questions of parsing address lists, etc. We do all that now in perl with
> send-email, where we can lean on some parsing libraries. So I dunno.

Hmmm.

> 
> > > If you're sending a long series, it's helpful to pre-populate various
> > > headers in the format-patch command with "--to", etc. I usually do so by
> > > sending the cover letter directly via mutt, and then using some perl
> > > hackery to convert those headers into format-patch args. The script I
> > 
> > Indeed, that hackery is what send-email already does, so how about
> > moving those features a bit upstream so that format-patch can do them
> > too?
> 
> Yeah, if they existed in format-patch I might be able to reuse them. I
> am hesitant, though, just because handling all the corner cases on
> parsing is going to be a bit of new C code.
> 
> > Although then, maybe it's simpler to teach send-email to learn to use
> > mutt(1) under the hood for the actual send.
> 
> I think you will find some corner cases in trying to make mutt act just
> like an mta accepting delivery. Two I can think of:
> 
>   1. It will take a body on stdin, but not a whole message. We can hack
>      around that with some postponed-folder magic, though.
> 
>   2. Bcc headers are stripped before sendmail sees the message (but
>      those addresses appear on the command-line). Converting that back
>      to bcc so that mutt can then re-strip them would be annoying but
>      possible. If you don't use bcc, it probably makes sense to just
>      punt on this.

Meh, I can live with no Bcc feature; I never used it before.  :)

> 
> So maybe a script like this:
> 
> -- >8 --
> #!/bin/sh
> 
> # ignore arguments; mutt will parse them itself
> # from to/cc headers. Note that we'll miss bcc this
> # way, but handling that would probably be kind of
> # tricky; we'd need to re-add those recipients as actual
> # bcc headers so that mutt knows how to handle them.
> 
> # spool the message to a fake mbox; we need to add
> # a "From" line to make it look legit
> trap 'rm -f to-send' 0 &&
> {
>   echo "From whatever Mon Sep 17 00:00:00 2001" &&
>   cat
> } >to-send &&

Would a named pipe work?  Or maybe we could use $(mktemp)?

> 
> # and then have mutt "resume" it. We have to redirect
> # stdin back from the terminal, since ours is a pipe
> # with the message contents.
> mutt -p \
>   -e 'set postponed=to-send' \
>   -e 'set edit_headers=yes' \
>   </dev/tty
> -- 8< --

Huh, this is magic sauce!  Works perfect for what I need.  This would
need to be packaged to the masses.  :-)

I found a minor problem: If I ctrl+C within mutt(1), I expect it to
cancel the last action, but this script intercepts the signal and exits.
We would probably need to ignore SIGINT from mutt-as-mta.

Other than that, this fulfills all of my needs.

Would you mind adding this as part of git?  Or should we suggest the
mutt project adding this script?


Thanks a lot!

Cheers,
Alex

> 
> and then in your git config:
> 
>   [sendemail]
>   sendmailcmd = /path/to/mutt-as-mta.sh
> 
> There are mutt-specific bits there that I don't think send-email should
> have to know about. Perhaps there are generic options that send-email
> could learn, but it really feels like you'd do better teaching mutt to
> be more ready to handle this (like taking a whole message on stdin,
> headers and all, rather than just a body).
> 
> -Peff

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Jeff King]

On Wed, Nov 08, 2023 at 10:02:52PM +0100, Alejandro Colomar wrote:

> > That seems like a reasonable feature. Probably it should be
> > --cc-from-trailer=signed-off-by, and then you could do the same with
> > other trailers.
> 
> That would work for me.  I only suggested the other one because it's
> aleady in send-email in that form.  But yeah, it might be useful to have
> finer control.  In my case, something that CCs every mail in the trailer
> would work (Reviewed-by, Suggested-by, ...  I want them all, always).

Oh, I didn't realize it existed in send-email. I am showing my ignorance
of it. ;)

> > # spool the message to a fake mbox; we need to add
> > # a "From" line to make it look legit
> > trap 'rm -f to-send' 0 &&
> > {
> >   echo "From whatever Mon Sep 17 00:00:00 2001" &&
> >   cat
> > } >to-send &&
> 
> Would a named pipe work?  Or maybe we could use $(mktemp)?

I suspect mutt wants it to be a real file. But yeah, mktemp would
definitely work. I actually started to write it that way but switched to
a static name for simplicity in demonstrating the idea. :)

One note, though. Later we need to pass this filename to mutt config:

> > mutt -p \
> >   -e 'set postponed=to-send' \

so it's a potential worry if "mktemp" might use a path with spaces or
funny characters (e.g., from $TMPDIR). Probably not much of a problem in
practice, though.

> Huh, this is magic sauce!  Works perfect for what I need.  This would
> need to be packaged to the masses.  :-)
> 
> I found a minor problem: If I ctrl+C within mutt(1), I expect it to
> cancel the last action, but this script intercepts the signal and exits.
> We would probably need to ignore SIGINT from mutt-as-mta.

Yeah, that might make sense, and can be done with trap.

> Would you mind adding this as part of git?  Or should we suggest the
> mutt project adding this script?

IMHO it is a little too weird and user-specific to really make sense in
either project. It's really glue-ing together two systems. And as it's
not something I use myself, I don't plan it moving it further along. But
you are welcome to take what I wrote and do what you will with it,
including submitting it to mutt.

-Peff

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2416 bytes --]

Hi Jeff,

On Wed, Nov 08, 2023 at 04:27:02PM -0500, Jeff King wrote:
> > > # spool the message to a fake mbox; we need to add
> > > # a "From" line to make it look legit
> > > trap 'rm -f to-send' 0 &&
> > > {
> > >   echo "From whatever Mon Sep 17 00:00:00 2001" &&
> > >   cat
> > > } >to-send &&
> > 
> > Would a named pipe work?  Or maybe we could use $(mktemp)?
> 
> I suspect mutt wants it to be a real file. But yeah, mktemp would
> definitely work. I actually started to write it that way but switched to
> a static name for simplicity in demonstrating the idea. :)
> 
> One note, though. Later we need to pass this filename to mutt config:
> 
> > > mutt -p \
> > >   -e 'set postponed=to-send' \
> 
> so it's a potential worry if "mktemp" might use a path with spaces or
> funny characters (e.g., from $TMPDIR). Probably not much of a problem in
> practice, though.
> 
> > Huh, this is magic sauce!  Works perfect for what I need.  This would
> > need to be packaged to the masses.  :-)
> > 
> > I found a minor problem: If I ctrl+C within mutt(1), I expect it to
> > cancel the last action, but this script intercepts the signal and exits.
> > We would probably need to ignore SIGINT from mutt-as-mta.
> 
> Yeah, that might make sense, and can be done with trap.

I've tried something even simpler:

---8<---
#!/bin/sh

mutt -H -;
--->8---

I used it for sending a couple of patches to linux-man@, and it seems to
work.  I don't have much experience with mutt, so maybe I'm missing some
corner cases.  Do you expect it to not work for some case?  Otherwise,
we might have a winner.  :)

> 
> > Would you mind adding this as part of git?  Or should we suggest the
> > mutt project adding this script?
> 
> IMHO it is a little too weird and user-specific to really make sense in
> either project. It's really glue-ing together two systems. And as it's
> not something I use myself, I don't plan it moving it further along. But
> you are welcome to take what I wrote and do what you will with it,
> including submitting it to mutt.

I'll start by creating a git repository in my own server, and will write
something about it to let the public know about it.  I'll also start
requiring contributors to linux-man@ to sign their patches, and
recommend them using this if they use mutt(1).

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Konstantin Ryabitsev]

On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:
> I used it for sending a couple of patches to linux-man@, and it seems to
> work.  I don't have much experience with mutt, so maybe I'm missing some
> corner cases.  Do you expect it to not work for some case?  Otherwise,
> we might have a winner.  :)

Since it's a Linux project, I suggest also checking out b4, which will do the
mail sending for you as part of the contributor-oriented features:

https://b4.docs.kernel.org/en/latest/contributor/overview.html

We also provide a web relay for people who can't configure or use SMTP due to
their company policies.

> > > Would you mind adding this as part of git?  Or should we suggest the
> > > mutt project adding this script?
> > 
> > IMHO it is a little too weird and user-specific to really make sense in
> > either project. It's really glue-ing together two systems. And as it's
> > not something I use myself, I don't plan it moving it further along. But
> > you are welcome to take what I wrote and do what you will with it,
> > including submitting it to mutt.
> 
> I'll start by creating a git repository in my own server, and will write
> something about it to let the public know about it.  I'll also start
> requiring contributors to linux-man@ to sign their patches, and
> recommend them using this if they use mutt(1).

B4 will also sign your patches for you. ;)

-K

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3697 bytes --]

Hi Konstantin,

On Thu, Nov 09, 2023 at 11:08:58AM -0500, Konstantin Ryabitsev wrote:
> On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:
> > I used it for sending a couple of patches to linux-man@, and it seems to
> > work.  I don't have much experience with mutt, so maybe I'm missing some
> > corner cases.  Do you expect it to not work for some case?  Otherwise,
> > we might have a winner.  :)
> 
> Since it's a Linux project, I suggest also checking out b4, which will do the
> mail sending for you as part of the contributor-oriented features:
> 
> https://b4.docs.kernel.org/en/latest/contributor/overview.html
> 
> We also provide a web relay for people who can't configure or use SMTP due to
> their company policies.
> 
> > > > Would you mind adding this as part of git?  Or should we suggest the
> > > > mutt project adding this script?
> > > 
> > > IMHO it is a little too weird and user-specific to really make sense in
> > > either project. It's really glue-ing together two systems. And as it's
> > > not something I use myself, I don't plan it moving it further along. But
> > > you are welcome to take what I wrote and do what you will with it,
> > > including submitting it to mutt.
> > 
> > I'll start by creating a git repository in my own server, and will write
> > something about it to let the public know about it.  I'll also start
> > requiring contributors to linux-man@ to sign their patches, and
> > recommend them using this if they use mutt(1).
> 
> B4 will also sign your patches for you. ;)

I haven't yet tried b4(1), and considered trying it some time ago, but
really didn't find git-send-email(1) and mutt(1) so difficult to use
that b4(1) would simplify much.  But still, I'll give it a chance.
Maybe I see why afterwards.

But I have tried patatt(1) before, which is what I think b4(1) uses for
signing.  Here are some of my concerns about patatt(4):

It doesn't sign the mail, but just the patch.  There's not much
difference, if any, in authenticability terms, but there's a big
difference in usability terms:

To authenticate a given patch submitted to a mailing list, the receiver
needs to also have patatt(1) configured.  Otherwise, it looks like a
random message.  MUAs normally don't show random headers (patatt(1)
signs by adding the signature header), so unless one is searching for
that header, it will be ignored.  This means, if I contribute to other
projects, I need to tell them my patch is signed via patatt(1) in
order for them to verify.  If instead, I sign the email as usual with my
MUA, every MUA will recognize the signature by default and show it to
the reader.

It also doesn't allow encrypting mail, so let's say I send some patch
fixing a security vulnerability, I'll need a custom tool to send it.  If
instead, I use mutt(1) to send it signed+encrypted to a mailing list
that provides a PGP public key, I can reuse my usual tools.

Also, and I don't know if b4(1) handles this automagically, but AFAIR,
patatt(1) didn't: fo signing a patch, I had to configure per-directory
with `patatt install-hook`.  I have more than a hundred git worktrees
(think of dozens of git repositories, and half a dozen worktrees --see
git-worktree(1)-- per repository).  If I need to configure every one of
those worktrees to sign all of my patches, that's going to be
cumbersome.  Also, I scrape and re-create worktrees for new branches
all the time, so I'd need to be installing hooks for patatt(1) all the
time.  Compare that to having mutt(1) configured once.  It doesn't
scale that well.

Cheers,
Alex

> 
> -K

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Konstantin Ryabitsev]

On Thu, Nov 09, 2023 at 06:42:19PM +0100, Alejandro Colomar wrote:
> I haven't yet tried b4(1), and considered trying it some time ago, but
> really didn't find git-send-email(1) and mutt(1) so difficult to use
> that b4(1) would simplify much.

Well, sending is only a small part of what b4 will do for you -- the core
benefits are really cover letter management, automatic versioning and
simplified list trailer collection. It's all tailored to kernel needs, but it
will work for any project that depends on mailed patches.

> But I have tried patatt(1) before, which is what I think b4(1) uses for
> signing.  Here are some of my concerns about patatt(4):
> 
> It doesn't sign the mail, but just the patch.

Well, no, it signs the entire thing, not just the patch, but it's true that
it's specifically targeted at patches (hence the name).

> There's not much
> difference, if any, in authenticability terms, but there's a big
> difference in usability terms:
> 
> To authenticate a given patch submitted to a mailing list, the receiver
> needs to also have patatt(1) configured.  Otherwise, it looks like a
> random message.

Yes, but that's a feature.

> MUAs normally don't show random headers (patatt(1)
> signs by adding the signature header), so unless one is searching for
> that header, it will be ignored.  This means, if I contribute to other
> projects, I need to tell them my patch is signed via patatt(1) in
> order for them to verify.  If instead, I sign the email as usual with my
> MUA, every MUA will recognize the signature by default and show it to
> the reader.

I go into this in the FAQ for patatt:
https://github.com/mricon/patatt#why-not-simply-pgp-sign-all-patches

Basically, developers really hated getting patches signed with PGP, either
inline or MIME, which is why it never took off. Putting it into the header
where it's not seen except by specialized tooling was a design choice.

> It also doesn't allow encrypting mail, so let's say I send some patch
> fixing a security vulnerability, I'll need a custom tool to send it.  If
> instead, I use mutt(1) to send it signed+encrypted to a mailing list
> that provides a PGP public key, I can reuse my usual tools.

Right, the goal was really *just* attestation. For encrypted patch exchange we
have remail (https://korg.docs.kernel.org/remail.html), which worked
significantly better than any other alternative we've considered.

> Also, and I don't know if b4(1) handles this automagically, but AFAIR,
> patatt(1) didn't: fo signing a patch, I had to configure per-directory
> with `patatt install-hook`.  I have more than a hundred git worktrees
> (think of dozens of git repositories, and half a dozen worktrees --see
> git-worktree(1)-- per repository).  If I need to configure every one of
> those worktrees to sign all of my patches, that's going to be
> cumbersome.  Also, I scrape and re-create worktrees for new branches
> all the time, so I'd need to be installing hooks for patatt(1) all the
> time.  Compare that to having mutt(1) configured once.  It doesn't
> scale that well.

Also true -- patatt was really envisioned as a library for b4, where you can
configure patch signing in your ~/.gitconfig for all projects.

-K

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 4055 bytes --]

Hi Konstantin,

On Thu, Nov 09, 2023 at 12:59:23PM -0500, Konstantin Ryabitsev wrote:
> On Thu, Nov 09, 2023 at 06:42:19PM +0100, Alejandro Colomar wrote:
> > I haven't yet tried b4(1), and considered trying it some time ago, but
> > really didn't find git-send-email(1) and mutt(1) so difficult to use
> > that b4(1) would simplify much.
> 
> Well, sending is only a small part of what b4 will do for you -- the core
> benefits are really cover letter management, automatic versioning and
> simplified list trailer collection. It's all tailored to kernel needs, but it
> will work for any project that depends on mailed patches.
> 
> > But I have tried patatt(1) before, which is what I think b4(1) uses for
> > signing.  Here are some of my concerns about patatt(4):
> > 
> > It doesn't sign the mail, but just the patch.
> 
> Well, no, it signs the entire thing, not just the patch, but it's true that
> it's specifically targeted at patches (hence the name).
> 
> > There's not much
> > difference, if any, in authenticability terms, but there's a big
> > difference in usability terms:
> > 
> > To authenticate a given patch submitted to a mailing list, the receiver
> > needs to also have patatt(1) configured.  Otherwise, it looks like a
> > random message.
> 
> Yes, but that's a feature.
> 
> > MUAs normally don't show random headers (patatt(1)
> > signs by adding the signature header), so unless one is searching for
> > that header, it will be ignored.  This means, if I contribute to other
> > projects, I need to tell them my patch is signed via patatt(1) in
> > order for them to verify.  If instead, I sign the email as usual with my
> > MUA, every MUA will recognize the signature by default and show it to
> > the reader.
> 
> I go into this in the FAQ for patatt:
> https://github.com/mricon/patatt#why-not-simply-pgp-sign-all-patches
> 
> Basically, developers really hated getting patches signed with PGP, either
> inline or MIME, which is why it never took off. Putting it into the header
> where it's not seen except by specialized tooling was a design choice.

It would be interesting if MUAs would support PGP signatures in a
header.  Did you consider that option back then?  Maybe patching a
mutt(1) or neomutt(1) to do that would have been simpler than developing
patatt(1).

> 
> > It also doesn't allow encrypting mail, so let's say I send some patch
> > fixing a security vulnerability, I'll need a custom tool to send it.  If
> > instead, I use mutt(1) to send it signed+encrypted to a mailing list
> > that provides a PGP public key, I can reuse my usual tools.
> 
> Right, the goal was really *just* attestation. For encrypted patch exchange we
> have remail (https://korg.docs.kernel.org/remail.html), which worked
> significantly better than any other alternative we've considered.
> 
> > Also, and I don't know if b4(1) handles this automagically, but AFAIR,
> > patatt(1) didn't: fo signing a patch, I had to configure per-directory
> > with `patatt install-hook`.  I have more than a hundred git worktrees
> > (think of dozens of git repositories, and half a dozen worktrees --see
> > git-worktree(1)-- per repository).  If I need to configure every one of
> > those worktrees to sign all of my patches, that's going to be
> > cumbersome.  Also, I scrape and re-create worktrees for new branches
> > all the time, so I'd need to be installing hooks for patatt(1) all the
> > time.  Compare that to having mutt(1) configured once.  It doesn't
> > scale that well.
> 
> Also true -- patatt was really envisioned as a library for b4, where you can
> configure patch signing in your ~/.gitconfig for all projects.

Overall, I think mutt(1) works better for me than patatt(1) via b4(1)
for crypto operations.  I don't use software that doesn't work with
PGP/MIME, and it's more versatile.  I'm still curious about the other
features of b4(1), so I'll try it for those.

Thanks!
Alex

> 
> -K

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Jeff King]

On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:

> I've tried something even simpler:
> 
> ---8<---
> #!/bin/sh
> 
> mutt -H -;
> --->8---
> 
> I used it for sending a couple of patches to linux-man@, and it seems to
> work.  I don't have much experience with mutt, so maybe I'm missing some
> corner cases.  Do you expect it to not work for some case?  Otherwise,
> we might have a winner.  :)

Wow, I don't know how I missed that when I read the manual. That was
exactly the feature I was thinking that mutt would need. ;)

So yeah, that is obviously better than the "postponed" hackery I showed.
I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that
Git adds to sendemail.sendmailcmd). So you can just invoke it directly
from your config like:

  git config sendemail.sendmailcmd "mutt -H -"

Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt
isn't willing to re-open the tty itself). But if you're happy with not
editing as they go through, then "-H" is then that's enough (in my
workflow, I do the final proofread via mutt).

-Peff

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1556 bytes --]

Hi Jeff,

On Thu, Nov 09, 2023 at 01:03:08PM -0500, Jeff King wrote:
> On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:
> 
> > I've tried something even simpler:
> > 
> > ---8<---
> > #!/bin/sh
> > 
> > mutt -H -;
> > --->8---
> > 
> > I used it for sending a couple of patches to linux-man@, and it seems to
> > work.  I don't have much experience with mutt, so maybe I'm missing some
> > corner cases.  Do you expect it to not work for some case?  Otherwise,
> > we might have a winner.  :)
> 
> Wow, I don't know how I missed that when I read the manual. That was
> exactly the feature I was thinking that mutt would need. ;)
> 
> So yeah, that is obviously better than the "postponed" hackery I showed.
> I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that
> Git adds to sendemail.sendmailcmd). So you can just invoke it directly
> from your config like:
> 
>   git config sendemail.sendmailcmd "mutt -H -"

Hmm great then!  Definitely a winner.  :)

> 
> Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt
> isn't willing to re-open the tty itself). But if you're happy with not
> editing as they go through, then "-H" is then that's enough (in my
> workflow, I do the final proofread via mutt).

Since git-send-email allows editing, I usually edit with that.  Having
-E would be redundant (and in fact it felt like that to me with your
suggested mutt-as-mta.sh) for my use case.

Cheers,
Alex

> 
> -Peff

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2469 bytes --]

Hi Jeff,

On Thu, Nov 09, 2023 at 01:03:08PM -0500, Jeff King wrote:
> On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:
> 
> > I've tried something even simpler:
> > 
> > ---8<---
> > #!/bin/sh
> > 
> > mutt -H -;
> > --->8---
> > 
> > I used it for sending a couple of patches to linux-man@, and it seems to
> > work.  I don't have much experience with mutt, so maybe I'm missing some
> > corner cases.  Do you expect it to not work for some case?  Otherwise,
> > we might have a winner.  :)
> 
> Wow, I don't know how I missed that when I read the manual. That was
> exactly the feature I was thinking that mutt would need. ;)
> 
> So yeah, that is obviously better than the "postponed" hackery I showed.
> I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that
> Git adds to sendemail.sendmailcmd). So you can just invoke it directly
> from your config like:
> 
>   git config sendemail.sendmailcmd "mutt -H -"

Having it directly in sendmailcmd causes some glitch: It repeats all CCs
in TO.  See a log:

	Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll): y
	OK. Log says:
	Sendmail: mutt -H - -i kevin@8t8.us mutt-dev@mutt.org alx@kernel.org e.sovetkin@gmail.com neomutt-devel@neomutt.org
	From: Alejandro Colomar <alx@kernel.org>
	To: Kevin McCarthy <kevin@8t8.us>,
		mutt-dev@mutt.org
	Cc: Alejandro Colomar <alx@kernel.org>,
		Jenya Sovetkin <e.sovetkin@gmail.com>,
		neomutt-devel@neomutt.org
	Subject: [PATCH] send.c: Allow crypto operations in batch and mailx modes.
	Date: Fri, 10 Nov 2023 01:41:24 +0100
	Message-ID: <20231110004128.5972-2-alx@kernel.org>
	X-Mailer: git-send-email 2.42.0
	MIME-Version: 1.0
	Content-Transfer-Encoding: 8bit

	Result: OK

The sent mail ended up being

	From: Alejandro Colomar <alx@kernel.org>
	To: Kevin McCarthy <kevin@8t8.us>, mutt-dev@mutt.org, alx@kernel.org,
		e.sovetkin@gmail.com, neomutt-devel@neomutt.org
	Cc: Alejandro Colomar <alx@kernel.org>,
		Jenya Sovetkin <e.sovetkin@gmail.com>, neomutt-devel@neomutt.org

So maybe we need the wrapper script to ignore the arguments.

Cheers,
Alex

> 
> Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt
> isn't willing to re-open the tty itself). But if you're happy with not
> editing as they go through, then "-H" is then that's enough (in my
> workflow, I do the final proofread via mutt).
> 
> -Peff

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2964 bytes --]

On Fri, Nov 10, 2023 at 01:51:34AM +0100, Alejandro Colomar wrote:
> Hi Jeff,
> 
> On Thu, Nov 09, 2023 at 01:03:08PM -0500, Jeff King wrote:
> > On Thu, Nov 09, 2023 at 04:26:23PM +0100, Alejandro Colomar wrote:
> > 
> > > I've tried something even simpler:
> > > 
> > > ---8<---
> > > #!/bin/sh
> > > 
> > > mutt -H -;
> > > --->8---
> > > 
> > > I used it for sending a couple of patches to linux-man@, and it seems to
> > > work.  I don't have much experience with mutt, so maybe I'm missing some
> > > corner cases.  Do you expect it to not work for some case?  Otherwise,
> > > we might have a winner.  :)
> > 
> > Wow, I don't know how I missed that when I read the manual. That was
> > exactly the feature I was thinking that mutt would need. ;)
> > 
> > So yeah, that is obviously better than the "postponed" hackery I showed.
> > I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that
> > Git adds to sendemail.sendmailcmd). So you can just invoke it directly
> > from your config like:
> > 
> >   git config sendemail.sendmailcmd "mutt -H -"
> 
> Having it directly in sendmailcmd causes some glitch: It repeats all CCs
> in TO.  See a log:
> 
> 	Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll): y
> 	OK. Log says:
> 	Sendmail: mutt -H - -i kevin@8t8.us mutt-dev@mutt.org alx@kernel.org e.sovetkin@gmail.com neomutt-devel@neomutt.org
> 	From: Alejandro Colomar <alx@kernel.org>
> 	To: Kevin McCarthy <kevin@8t8.us>,
> 		mutt-dev@mutt.org
> 	Cc: Alejandro Colomar <alx@kernel.org>,
> 		Jenya Sovetkin <e.sovetkin@gmail.com>,
> 		neomutt-devel@neomutt.org
> 	Subject: [PATCH] send.c: Allow crypto operations in batch and mailx modes.
> 	Date: Fri, 10 Nov 2023 01:41:24 +0100
> 	Message-ID: <20231110004128.5972-2-alx@kernel.org>
> 	X-Mailer: git-send-email 2.42.0
> 	MIME-Version: 1.0
> 	Content-Transfer-Encoding: 8bit
> 
> 	Result: OK
> 
> The sent mail ended up being
> 
> 	From: Alejandro Colomar <alx@kernel.org>
> 	To: Kevin McCarthy <kevin@8t8.us>, mutt-dev@mutt.org, alx@kernel.org,
> 		e.sovetkin@gmail.com, neomutt-devel@neomutt.org
> 	Cc: Alejandro Colomar <alx@kernel.org>,
> 		Jenya Sovetkin <e.sovetkin@gmail.com>, neomutt-devel@neomutt.org
> 
> So maybe we need the wrapper script to ignore the arguments.

Heh!  The following trick works as well, without needing a script:

[sendemail]
	sendmailcmd = mutt -H - && true

It probably relies too much on git-send-email(1)'s current
implementation, but it works.  :)

Cheers,
Alex

> 
> Cheers,
> Alex
> 
> > 
> > Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt
> > isn't willing to re-open the tty itself). But if you're happy with not
> > editing as they go through, then "-H" is then that's enough (in my
> > workflow, I do the final proofread via mutt).
> > 
> > -Peff
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-send-email: Send with mutt(1)

[Jeff King]

On Fri, Nov 10, 2023 at 02:30:49PM +0100, Alejandro Colomar wrote:

> > Having it directly in sendmailcmd causes some glitch: It repeats all CCs
> > in TO.  See a log:

Ah, right. That makes sense. send-email has to pass all of the envelope
recipients on the command-line, which mutt then interprets as
destinations to add to "to". Mutt is smart enough to de-duplicate the
"to", but it (correctly) allows duplicate to/cc. (My basic test didn't
have any cc's).

> > So maybe we need the wrapper script to ignore the arguments.
> 
> Heh!  The following trick works as well, without needing a script:
> 
> [sendemail]
> 	sendmailcmd = mutt -H - && true
> 
> It probably relies too much on git-send-email(1)'s current
> implementation, but it works.  :)

I was going to suggest the same thing. And no, I don't think it is
relying on any send-email implementation details. Git always tries to
feed command-invoking config like this to the shell for consistency (and
to allow tricks like this).

-Peff

Re: git-send-email: Send with mutt(1)

[Junio C Hamano]

Alejandro Colomar <alx@kernel.org> writes:

> [sendemail]
> 	sendmailcmd = mutt -H - && true

Ah, this is fun, an ugly but serviceable trick to ignore the command
line arguments.  I briefly wondered it would work equally well and
much more readable to simply append "#", but the above should work
just fine.