[b4][PATCH 0/2] Improvements to DKIM signature verification

[b4][PATCH 0/2] Improvements to DKIM signature verification

[Paul Barker]

Testing 'b4 am' with patches sent via Office 365 to lists.sr.ht showed
the same issues I found with patatt in handling re-encoded headers. So
there's a patch here to fix this and a patch to improve the ability to
investigate DKIM verification failures with debug output.

NOTE: The attestation key on these patch emails is different to the one
I used on previous patches - I've generated a new key so I've got
separate identities for 'work' and 'personal' hats. You may need to
referesh keys from keys.openpgp.org.

Paul Barker (2):
  Handle MIME encoded-word in DKIM-Signature headers
  Include dkim log output when -d/--debug argument is passed

 b4/__init__.py | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

-- 
2.31.1

[b4][PATCH 1/2] Handle MIME encoded-word in DKIM-Signature headers

[Paul Barker]

As recently found in patatt [1], mail gateways and archivers may mangle
headers like DKIM-Signature if they are sent as an excessively long
line. An example of this occuring was found when the DKIM-Signature
header generated by Microsoft Office 365 collided with the
header re-encoding performed by lists.sr.ht when generating mbox
archive files. This encoding causes dkim.verify() to fail.

The Python email.header module provides the decode_header() and
make_header() functions which can be used to handle MIME encoded-word
syntax or other header manglings which may occur. Fixing up the header
content using these functions before calling dkim.verify() allows the
verification to succeed.

[1]: https://lore.kernel.org/tools/20210531140539.7630-1-paul@pbarker.dev/

Signed-off-by: Paul Barker <paul@pbarker.dev>
---
 b4/__init__.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/b4/__init__.py b/b4/__init__.py
index a163364..168b722 100644
--- a/b4/__init__.py
+++ b/b4/__init__.py
@@ -1006,6 +1006,10 @@ class LoreMessage:
 
         seenatts = list()
         for hn, hval in dkhdrs:
+            # Handle MIME encoded-word syntax or other types of header encoding if
+            # present.
+            if '?q?' in hval:
+                hval = str(email.header.make_header(email.header.decode_header(hval)))
             errors = list()
             hdata = LoreMessage.get_parts_from_header(hval)
             logger.debug('Loading DKIM attestation for d=%s, s=%s', hdata['d'], hdata['s'])
-- 
2.31.1

[b4][PATCH 2/2] Include dkim log output when -d/--debug argument is passed

[Paul Barker]

We can pass a logger object to dkim.verify() which will be used to
report internal errors and debugging info. This can be helpful when
investigating DKIM verification issues but is probably not wanted during
normal operation so the log level of each message is reset to DEBUG.
Each message is also prefixed with 'DKIM: ' to identify its origin when
debug output is enabled.

Signed-off-by: Paul Barker <paul@pbarker.dev>
---
 b4/__init__.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/b4/__init__.py b/b4/__init__.py
index 168b722..9721e22 100644
--- a/b4/__init__.py
+++ b/b4/__init__.py
@@ -47,6 +47,17 @@ __VERSION__ = '0.8-dev'
 
 logger = logging.getLogger('b4')
 
+def _dkim_log_filter(record):
+    # Hide all dkim logging output in normal operation by setting the level to
+    # DEBUG. If debugging output has been enabled then prefix dkim logging
+    # output to make its origin clear.
+    record.levelno = logging.DEBUG
+    record.levelname = 'DEBUG'
+    record.msg = 'DKIM: ' + record.msg
+    return True
+dkimlogger = logger.getChild('dkim')
+dkimlogger.addFilter(_dkim_log_filter)
+
 HUNK_RE = re.compile(r'^@@ -\d+(?:,(\d+))? \+\d+(?:,(\d+))? @@')
 FILENAME_RE = re.compile(r'^(---|\+\+\+) (\S+)')
 
@@ -1028,7 +1039,7 @@ class LoreMessage:
                     signtime = self.date
 
             self.msg._headers.append((hn, hval))  # noqa
-            res = dkim.verify(self.msg.as_bytes())
+            res = dkim.verify(self.msg.as_bytes(), logger=dkimlogger)
 
             attestor = LoreAttestorDKIM(res, identity, signtime, errors)
             logger.debug('DKIM verify results: %s=%s', identity, res)
-- 
2.31.1

Re: [b4][PATCH 0/2] Improvements to DKIM signature verification

[Konstantin Ryabitsev]

On Mon, 7 Jun 2021 11:02:50 +0100, Paul Barker wrote:
> Testing 'b4 am' with patches sent via Office 365 to lists.sr.ht showed
> the same issues I found with patatt in handling re-encoded headers. So
> there's a patch here to fix this and a patch to improve the ability to
> investigate DKIM verification failures with debug output.
> [...]

Applied, thanks!

[1/2] Handle MIME encoded-word in DKIM-Signature headers
      commit: 366bcd1fd8a10b6125e591f0c35cc85b10c9c950
[2/2] Include dkim log output when -d/--debug argument is passed
      commit: 506cf91716ff32c3747829b8556ce2d26b1936a4

Best regards,
-- 
Konstantin Ryabitsev <konstantin@linuxfoundation.org>