[PATCH 0/2] A couple of fixes for PATCHTOOL = "git"

[PATCH 0/2] A couple of fixes for PATCHTOOL = "git"

[Paul Eggleton]

A couple of fixes for PATCHTOOL = "git" (which devtool makes extensive
use of).


The following changes since commit 7e0f95bf359bc3b5bb1578024a993e184de155cd:

  base.bbclass: Drop unnecessary dirs setting (2016-09-22 11:08:23 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/patch-git-fixes
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/patch-git-fixes

Paul Eggleton (2):
  lib/oe/patch: exclude "From <hash>" from commit message when PATCHTOOL
    is "git"
  lib/oe/patch: improve accuracy of patch header extraction

 meta/lib/oe/patch.py | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)

-- 
2.5.5

[PATCH 1/2] lib/oe/patch: exclude "From <hash>" from commit message when PATCHTOOL is "git"

[Paul Eggleton]

If you leave "From <hash>" lines in the commit message it can actually
break git rebase because it tries to interpret the line in the context
of the current repository, and if the hash is invalid then a rebase
will blow up with:

  fatal: git cat-file: could not get object info

or in newer git versions:

  error: unable to find <hash>
  fatal: git cat-file <hash>: bad file

(I hit this when I tried to do a devtool upgrade on openssl to 1.0.2i
the first time I did "git rebase --skip")

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/lib/oe/patch.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 05e0faa..c04f098 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -317,6 +317,7 @@ class GitApplyTree(PatchTree):
     def interpretPatchHeader(headerlines):
         import re
         author_re = re.compile('[\S ]+ <\S+@\S+\.\S+>')
+        from_commit_re = re.compile('^From [a-z0-9]{40} .*')
         outlines = []
         author = None
         date = None
@@ -346,6 +347,9 @@ class GitApplyTree(PatchTree):
                 # git is fussy about author formatting i.e. it must be Name <email@domain>
                 if author_re.match(authorval):
                     author = authorval
+            elif from_commit_re.match(line):
+                # We don't want the From <commit> line - if it's present it will break rebasing
+                continue
             outlines.append(line)
         return outlines, author, date, subject
 
-- 
2.5.5

[PATCH 2/2] lib/oe/patch: improve accuracy of patch header extraction

[Paul Eggleton]

When PATCHTOOL = "git", if we need to manually apply a patch and then
commit it (i.e. when git am doesn't work) we try to extract the author /
date / shortlog from the patch header. Make the following improvements
to that extraction process:

* If there's no explicit Subject: but the first line is followed by a
  blank line, isn't an Upstream-Status: or Index: marker and isn't too
  long, then assume it's good enough to be the shortlog. This avoids
  having too many patches with "Upgrade to version x.y" as the shortlog
  (since that is often when patches get added).
* Add --follow to the command we use to find the commit that added the
  patch, so we mostly get the commit that added the patch rather than
  getting stuck on upgrade commits that last moved/renamed the patch
* Populate the date from the commit that added the patch if we were able
  to get the author but not the date from the patch (otherwise you get
  today's date which is less useful).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/lib/oe/patch.py | 28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index c04f098..0332f10 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -351,6 +351,21 @@ class GitApplyTree(PatchTree):
                 # We don't want the From <commit> line - if it's present it will break rebasing
                 continue
             outlines.append(line)
+
+        if not subject:
+            firstline = None
+            for line in headerlines:
+                line = line.strip()
+                if firstline:
+                    if line:
+                        # Second line is not blank, the first line probably isn't usable
+                        firstline = None
+                    break
+                elif line:
+                    firstline = line
+            if firstline and not firstline.startswith(('#', 'Index:', 'Upstream-Status:')) and len(firstline) < 100:
+                subject = firstline
+
         return outlines, author, date, subject
 
     @staticmethod
@@ -373,21 +388,24 @@ class GitApplyTree(PatchTree):
         # Process patch header and extract useful information
         lines = GitApplyTree.extractPatchHeader(patchfile)
         outlines, author, date, subject = GitApplyTree.interpretPatchHeader(lines)
-        if not author or not subject:
+        if not author or not subject or not date:
             try:
-                shellcmd = ["git", "log", "--format=email", "--diff-filter=A", "--", patchfile]
+                shellcmd = ["git", "log", "--format=email", "--follow", "--diff-filter=A", "--", patchfile]
                 out = runcmd(["sh", "-c", " ".join(shellcmd)], os.path.dirname(patchfile))
             except CmdError:
                 out = None
             if out:
                 _, newauthor, newdate, newsubject = GitApplyTree.interpretPatchHeader(out.splitlines())
-                if not author or not date:
-                    # These really need to go together
+                if not author:
+                    # If we're setting the author then the date should be set as well
                     author = newauthor
                     date = newdate
+                elif not date:
+                    # If we don't do this we'll get the current date, at least this will be closer
+                    date = newdate
                 if not subject:
                     subject = newsubject
-        if subject:
+        if subject and outlines and not outlines[0].strip() == subject:
             outlines.insert(0, '%s\n\n' % subject.strip())
 
         # Write out commit message to a file
-- 
2.5.5