Re: [patatt][PATCH] Handle MIME encoded-word & other header manglings

[Paul Barker <paul@pbarker.dev>]

On Mon, 31 May 2021 09:04:59 -0400
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

> On Sun, May 30, 2021 at 05:26:25PM +0100, Paul Barker wrote:
> > When testing patatt with patches sent to a sr.ht hosted mailing
> > list, it was found that long header lines (such as the
> > X-Developer-Signature line) were re-encoded using the MIME
> > encoded-word syntax (RFC 2047) when an mbox archive is generated,
> > causing patatt to choke on the resulting text which looks like this:
> > 
> >     X-Developer-Signature: v=1; a=openpgp-sha256; l=672;
> > h=from:subject; bh=C40yOKgIfnNIUP+OW9WyPdBfljkZPpfUL1NepOODlx8=;
> > =?utf-8?q?b=3DowGbwMvMwCF2?=
> > =?utf-8?q?w7xIXuiX9CvG02pJDAmb67lTNi0+IeF97TL76vtKD7xjSjaluz0o/KfmZLX8rMi7_?=
> > =?utf-8?q?l3M6O0pZGMQ4GGTFFFl2z951+fqDJVt7b0gHw8xhZQIZwsDFKQATydFhZJi+fFfvJ?=
> > =?utf-8?q?8+0MF7GrfzWnP?=
> > K7mAM/3n/r/UC+bprf6/g114QYGdbHcsaK7b1nanfA4IeZi1V0lL26cruXUWxgSEnNDP1FrAA=
> >  
> 
> Bah. I understand that we must handle this situation, but this is an
> annoying and unncessary hack when WSP characters are already
> conveniently present in the header.
> 
> > Avoiding this issue by neatly wrapping the X-Developer-Signature
> > header before sending doesn't appear to be possible without making
> > invasive changes to git-send-email and/or the Net::SMTP perl
> > module. The header content generated by patatt is wrapped at 78
> > characters as can be seen here from a locally signed patch file:
> > 
> >     X-Developer-Signature: v=1; a=openpgp-sha256; l=672;
> > h=from:subject; bh=C40yOKgIfnNIUP+OW9WyPdBfljkZPpfUL1NepOODlx8=;
> >     b=owGbwMvMwCF2w7xIXuiX9CvG02pJDAmbN1xO2bT4hIT3tcvsq+8rPfCOKdmU7vag8J+ak9XysyLv
> >     Xs7p7ChlYRDjYJAVU2TZPXvX5esPlmztvSEdDDOHlQlkCAMXpwBMpG0Dw/9Kpzgpc8UsQwOPK/taW6
> >     dFnZyy5QlXPfNCC4WTc76ft9ZnZJjI37a17fP7sxvclKJ1tm36EhITcK62Pphje9KrmOxMJg4A
> > 
> > Running `git send-email --smtp-debug=1 0001.patch` shows that this
> > is joined into a single long line before the message is sent:
> >   
> >     Net::SMTP::_SSL=GLOB(0x5646fbdc3ac8)>>> X-Developer-Signature:
> > v=1; a=openpgp-sha256; l=672; h=from:subject;
> > bh=C40yOKgIfnNIUP+OW9WyPdBfljkZPpfUL1NepOODlx8=;
> > b=owGbwMvMwCF2w7xIXuiX9CvG02pJDAmb571P2bT4hIT3tcvsq+8rPfCOKdmU7vag8J+ak9XysyLv
> > Xs7p7ChlYRDjYJAVU2TZPXvX5esPlmztvSEdDDOHlQlkCAMXpwBM5JA3I8O5hP6Tqm7lJst0rldcux
> > 1V7M4q8T5o1fPU6Zs+hxj+SjvN8D/DK3rn8b0m34/Xy388Yeu8jvFdJf/c6Y6LDU7Hulj01nAAAA==
> >  
> 
> This, too, bugs me to no end. :) I've actually considered making
> --hook not perform any wrapping, but I'm hoping that git-send-email
> can be fixed to not do this. To be compliant with (admittedly old and
> obsoleted) email RFCs, headers must be no longer than 78 characters,
> so I'm pretty sure what git-send-email does is an unintentional
> side-effect.
> 
> > So we need to accept that the X-Developer-Signature line may be
> > quite long and so may be re-encoded by a mail server or archiver.  
> 
> Agreed, thanks for finding this.
> 
> >      @staticmethod
> >      def _dkim_canonicalize_header(hval: bytes) -> bytes:  
> 
> I think it would make sense to wrap this around in:
> 
> if hval.find(b'?q?') >= 0:
> 
> This will save unnecessary decoding/encoding/header parsing. Do you
> agree?
> 

Agreed.

> > +        # The decode_header() function we're about to call
> > requires a str
> > +        # argument. Since RFC2822 (sec 2.2), header fields must be
> > ASCII
> > +        # characters so this is easy to achieve.
> > +        hval = hval.decode('ascii')  
> 
> I think we'll want to pass errors='ignore' here just to avoid a
> backtrace for obviously wrong header values.

Agreed.

I'll send a v2.

Thanks,

-- 
Paul Barker
https://pbarker.dev/