[OE-core][PATCH v6 0/4] IPK signing for the gpg_sign module

[OE-core][PATCH v6 0/4] IPK signing for the gpg_sign module

[Ioan-Adrian Ratiu]

This patch series extends the gpg_sign module to support ipk signing.

v6 implements Markus' feedback. The most notable change is the sign_ipk
and detach_sign merger, as they were almost identical in functionality.
This also meant a refactoring for detach_sign and a bug fix for the
STDIN file descriptor introduced in gpg > 2.1.

Technically that STDIN bug is a feature (meh) of gpg >2.1 which breaks
existing behaviour so we have to work around it i.e. check the gpg
version and use the loopback interface. This means that gpg-agent to
which gpg >2.1 always connects needs to be running permanently.

Ioan-Adrian Ratiu (4):
  gpg_sign: add local ipk package signing functionality
  gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor
  gpg_sign: export_pubkey: add signature type support
  package_manager: sign IPK package feeds

 meta/classes/package_ipk.bbclass       |  5 +++
 meta/classes/sign_ipk.bbclass          | 52 ++++++++++++++++++++++++
 meta/classes/sign_package_feed.bbclass | 12 +++++-
 meta/lib/oe/gpg_sign.py                | 74 +++++++++++++++++++++++++++-------
 meta/lib/oe/package_manager.py         | 17 +++++++-
 5 files changed, 143 insertions(+), 17 deletions(-)
 create mode 100644 meta/classes/sign_ipk.bbclass

-- 
2.7.1

[OE-core][PATCH v6 1/4] gpg_sign: add local ipk package signing functionality

[Ioan-Adrian Ratiu]

Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
module and configure signing similar to how rpm does it. sign_ipk uses
gpg_sign's detach_sign because its functionality is identical to package
feed signing.

IPK signing process is a bit different from rpm:
    - Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
    - Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
    - Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/classes/package_ipk.bbclass |  5 ++++
 meta/classes/sign_ipk.bbclass    | 52 ++++++++++++++++++++++++++++++++++++++++
 meta/lib/oe/gpg_sign.py          | 50 ++++++++++++++++++++++++++++----------
 3 files changed, 94 insertions(+), 13 deletions(-)
 create mode 100644 meta/classes/sign_ipk.bbclass

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 51bee28..f64837a 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -246,6 +246,11 @@ python do_package_ipk () {
             bb.utils.unlockfile(lf)
             raise bb.build.FuncFailed("opkg-build execution failed")
 
+        if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
+            ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
+            ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
+            sign_ipk(d, ipk_to_sign)
+
         cleanupcontrol(root)
         bb.utils.unlockfile(lf)
 
diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
new file mode 100644
index 0000000..a481f6d
--- /dev/null
+++ b/meta/classes/sign_ipk.bbclass
@@ -0,0 +1,52 @@
+# Class for generating signed IPK packages.
+#
+# Configuration variables used by this class:
+# IPK_GPG_PASSPHRASE_FILE
+#           Path to a file containing the passphrase of the signing key.
+# IPK_GPG_NAME
+#           Name of the key to sign with.
+# IPK_GPG_BACKEND
+#           Optional variable for specifying the backend to use for signing.
+#           Currently the only available option is 'local', i.e. local signing
+#           on the build host.
+# IPK_GPG_SIGNATURE_TYPE
+#           Optional variable for specifying the type of gpg signatures, can be:
+#                     1. Ascii armored (ASC), default if not set
+#                     2. Binary (BIN)
+# GPG_BIN
+#           Optional variable for specifying the gpg binary/wrapper to use for
+#           signing.
+# GPG_PATH
+#           Optional variable for specifying the gnupg "home" directory:
+#
+
+inherit sanity
+
+IPK_SIGN_PACKAGES = '1'
+IPK_GPG_BACKEND ?= 'local'
+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
+
+python () {
+    # Check configuration
+    for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
+        if not d.getVar(var, True):
+            raise_sanity_error("You need to define %s in the config" % var, d)
+
+    sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
+    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+        raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+}
+
+def sign_ipk(d, ipk_to_sign):
+    from oe.gpg_sign import get_signer
+
+    bb.debug(1, 'Signing ipk: %s' % ipk_to_sign)
+
+    signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
+    sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
+    is_ascii_sig = (sig_type.upper() != "BIN")
+
+    signer.detach_sign(ipk_to_sign,
+                       d.getVar('IPK_GPG_NAME', True),
+                       d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
+                       armor=is_ascii_sig)
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ada1b2f..ef47d1a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -1,5 +1,6 @@
 """Helper module for GPG signing"""
 import os
+import sys
 
 import bb
 import oe.utils
@@ -50,6 +51,7 @@ class LocalSigner(object):
             bb.error('rpmsign failed: %s' % proc.before.strip())
             raise bb.build.FuncFailed("Failed to sign RPM packages")
 
+
     def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
         """Create a detached signature of a file"""
         import subprocess
@@ -57,23 +59,45 @@ class LocalSigner(object):
         if passphrase_file and passphrase:
             raise Exception("You should use either passphrase_file of passphrase, not both")
 
-        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty', '--yes',
-               '-u', keyid]
-        if passphrase_file:
-            cmd += ['--passphrase-file', passphrase_file]
-        else:
-            cmd += ['--passphrase-fd', '0']
+        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty', '--yes', '-u', keyid]
+
         if self.gpg_path:
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
-        cmd.append(input_file)
-        job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-                               stderr=subprocess.PIPE)
-        _, stderr = job.communicate(passphrase)
-        if job.returncode:
-            raise bb.build.FuncFailed("Failed to create signature for '%s': %s" %
-                                      (input_file, stderr))
+
+        try:
+            keypipe = os.pipe()
+
+            if passphrase_file:
+                with open(passphrase_file) as fobj:
+                    os.write(keypipe[1], fobj.readline());
+            else:
+                os.write(keypipe[1], passphrase)
+
+            cmd += ["--passphrase-fd",  str(keypipe[0])]
+            cmd += [input_file]
+
+            job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+            (_, stderr) = job.communicate(passphrase)
+
+            os.close(keypipe[1])
+            os.close(keypipe[0])
+
+            if job.returncode:
+                raise bb.build.FuncFailed("GPG exited with code %d: %s" %
+                                          (job.returncode, stderr))
+
+        except IOError as e:
+            bb.error("IO error (%s): %s" % (e.errno, e.strerror))
+            raise Exception("Failed to sign '%s'" % input_file)
+        except OSError as e:
+            bb.error("OS error (%s): %s" % (e.errno, e.strerror))
+            raise Exception("Failed to sign '%s" % input_file)
+        except:
+            bb.error("Unexpected error (%s): %s" % (sys.exc_info()[0], sys.exc_info()[1]))
+            raise Exception("Failed to sign '%s'" % input_file)
+
 
     def verify(self, sig_file):
         """Verify signature"""
-- 
2.7.1

[OE-core][PATCH v6 2/4] gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor

[Ioan-Adrian Ratiu]

Starting from v2.1 passing passwords directly to gpg does not work
anymore [1], instead a loopback interface must be used otherwise
gpg >2.1 will error out with:
"gpg: signing failed: Inappropriate ioctl for device"

gpg <2.1 does not work with the new --pinentry-mode arg and gives an
invalid option error, so we detect what is the running version of gpg
and pass it accordingly.

[1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/lib/oe/gpg_sign.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ef47d1a..9f6b0f0 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -66,6 +66,12 @@ class LocalSigner(object):
         if armor:
             cmd += ['--armor']
 
+        #gpg > 2.1 supports password pipes only through the loopback interface
+        #gpg < 2.1 errors out if given unknown parameters
+        gpg_ver = self.get_gpg_version()
+        if gpg_ver > 2.1:
+            cmd += ['--pinentry-mode', 'loopback']
+
         try:
             keypipe = os.pipe()
 
@@ -99,6 +105,20 @@ class LocalSigner(object):
             raise Exception("Failed to sign '%s'" % input_file)
 
 
+    def get_gpg_version(self):
+        """Return the gpg version"""
+        import subprocess
+
+        job = subprocess.Popen([self.gpg_bin, "--version"], stdout=subprocess.PIPE)
+        (stdout, _) = job.communicate()
+
+        if job.returncode:
+            raise bb.build.FuncFailed("Could not get gpg version (is %s installed?)" %
+                                      self.gpg_bin)
+
+        return stdout.split()[2]
+
+
     def verify(self, sig_file):
         """Verify signature"""
         cmd = self.gpg_bin + " --verify "
-- 
2.7.1

[OE-core][PATCH v6 3/4] gpg_sign: export_pubkey: add signature type support

[Ioan-Adrian Ratiu]

Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/lib/oe/gpg_sign.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 9f6b0f0..9b3864a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -13,12 +13,14 @@ class LocalSigner(object):
         self.gpg_path = d.getVar('GPG_PATH', True)
         self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
 
-    def export_pubkey(self, output_file, keyid):
+    def export_pubkey(self, output_file, keyid, armor=True):
         """Export GPG public key to a file"""
-        cmd = '%s --batch --yes --export --armor -o %s ' % \
+        cmd = '%s --batch --yes --export -o %s ' % \
                 (self.gpg_bin, output_file)
         if self.gpg_path:
             cmd += "--homedir %s " % self.gpg_path
+        if armor:
+            cmd += "--armor "
         cmd += keyid
         status, output = oe.utils.getstatusoutput(cmd)
         if status:
-- 
2.7.1

[OE-core][PATCH v6 4/4] package_manager: sign IPK package feeds

[Ioan-Adrian Ratiu]

Create gpg signed ipk package feeds using the gpg backend if configured.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/classes/sign_package_feed.bbclass | 12 +++++++++++-
 meta/lib/oe/package_manager.py         | 17 +++++++++++++++--
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass
index 63ca02f..85c37f3 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -10,6 +10,12 @@
 #           Optional variable for specifying the backend to use for signing.
 #           Currently the only available option is 'local', i.e. local signing
 #           on the build host.
+# PACKAGE_FEED_GPG_SIGNATURE_TYPE
+#           Optional variable for specifying the type of gpg signature, can be:
+#               1. Ascii armored (ASC), default if not set
+#               2. Binary (BIN)
+#           This variable is only available for IPK feeds. It is ignored on
+#           other packaging backends.
 # GPG_BIN
 #           Optional variable for specifying the gpg binary/wrapper to use for
 #           signing.
@@ -20,7 +26,7 @@ inherit sanity
 
 PACKAGE_FEED_SIGN = '1'
 PACKAGE_FEED_GPG_BACKEND ?= 'local'
-
+PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC'
 
 python () {
     # Check sanity of configuration
@@ -28,6 +34,10 @@ python () {
         if not d.getVar(var, True):
             raise_sanity_error("You need to define %s in the config" % var, d)
 
+    sigtype = d.getVar("PACKAGE_FEED_GPG_SIGNATURE_TYPE", True)
+    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+        raise_sanity_error("Bad value for PACKAGE_FEED_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+
     # Set expected location of the public key
     d.setVar('PACKAGE_FEED_GPG_PUBKEY',
              os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', False),
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index b30a4da..606ba24 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -163,11 +163,16 @@ class OpkgIndexer(Indexer):
                      "MULTILIB_ARCHS"]
 
         opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
 
         if not os.path.exists(os.path.join(self.deploy_dir, "Packages")):
             open(os.path.join(self.deploy_dir, "Packages"), "w").close()
 
         index_cmds = []
+        index_sign_files = []
         for arch_var in arch_vars:
             archs = self.d.getVar(arch_var, True)
             if archs is None:
@@ -186,6 +191,8 @@ class OpkgIndexer(Indexer):
                 index_cmds.append('%s -r %s -p %s -m %s' %
                                   (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
 
+                index_sign_files.append(pkgs_file)
+
         if len(index_cmds) == 0:
             bb.note("There are no packages in %s!" % self.deploy_dir)
             return
@@ -193,9 +200,15 @@ class OpkgIndexer(Indexer):
         result = oe.utils.multiprocess_exec(index_cmds, create_index)
         if result:
             bb.fatal('%s' % ('\n'.join(result)))
-        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
-            raise NotImplementedError('Package feed signing not implementd for ipk')
 
+        if signer:
+            feed_sig_type = self.d.getVar('PACKAGE_FEED_GPG_SIGNATURE_TYPE', True)
+            is_ascii_sig = (feed_sig_type.upper() != "BIN")
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   armor=is_ascii_sig)
 
 
 class DpkgIndexer(Indexer):
-- 
2.7.1

[PATCH v6 0/4] IPK signing for the gpg_sign module

[Ioan-Adrian Ratiu]

This patch series extends the gpg_sign module to support ipk signing.

v6 implements Markus' feedback. The most notable change is the sign_ipk
and detach_sign merger, as they were almost identical in functionality.
This also meant a refactoring for detach_sign and a bug fix for the
STDIN file descriptor introduced in gpg > 2.1.

Technically that STDIN bug is a feature (meh) of gpg >2.1 which breaks
existing behaviour so we have to work around it i.e. check the gpg
version and use the loopback interface. This means that gpg-agent to
which gpg >2.1 always connects needs to be running permanently.

Ioan-Adrian Ratiu (4):
  gpg_sign: add local ipk package signing functionality
  gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor
  gpg_sign: export_pubkey: add signature type support
  package_manager: sign IPK package feeds

 meta/classes/package_ipk.bbclass       |  5 +++
 meta/classes/sign_ipk.bbclass          | 52 ++++++++++++++++++++++++
 meta/classes/sign_package_feed.bbclass | 12 +++++-
 meta/lib/oe/gpg_sign.py                | 74 +++++++++++++++++++++++++++-------
 meta/lib/oe/package_manager.py         | 17 +++++++-
 5 files changed, 143 insertions(+), 17 deletions(-)
 create mode 100644 meta/classes/sign_ipk.bbclass

-- 
2.7.1

[PATCH v6 1/4] gpg_sign: add local ipk package signing functionality

[Ioan-Adrian Ratiu]

Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
module and configure signing similar to how rpm does it. sign_ipk uses
gpg_sign's detach_sign because its functionality is identical to package
feed signing.

IPK signing process is a bit different from rpm:
    - Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
    - Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
    - Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/classes/package_ipk.bbclass |  5 ++++
 meta/classes/sign_ipk.bbclass    | 52 ++++++++++++++++++++++++++++++++++++++++
 meta/lib/oe/gpg_sign.py          | 50 ++++++++++++++++++++++++++++----------
 3 files changed, 94 insertions(+), 13 deletions(-)
 create mode 100644 meta/classes/sign_ipk.bbclass

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 51bee28..f64837a 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -246,6 +246,11 @@ python do_package_ipk () {
             bb.utils.unlockfile(lf)
             raise bb.build.FuncFailed("opkg-build execution failed")
 
+        if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
+            ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
+            ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
+            sign_ipk(d, ipk_to_sign)
+
         cleanupcontrol(root)
         bb.utils.unlockfile(lf)
 
diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
new file mode 100644
index 0000000..a481f6d
--- /dev/null
+++ b/meta/classes/sign_ipk.bbclass
@@ -0,0 +1,52 @@
+# Class for generating signed IPK packages.
+#
+# Configuration variables used by this class:
+# IPK_GPG_PASSPHRASE_FILE
+#           Path to a file containing the passphrase of the signing key.
+# IPK_GPG_NAME
+#           Name of the key to sign with.
+# IPK_GPG_BACKEND
+#           Optional variable for specifying the backend to use for signing.
+#           Currently the only available option is 'local', i.e. local signing
+#           on the build host.
+# IPK_GPG_SIGNATURE_TYPE
+#           Optional variable for specifying the type of gpg signatures, can be:
+#                     1. Ascii armored (ASC), default if not set
+#                     2. Binary (BIN)
+# GPG_BIN
+#           Optional variable for specifying the gpg binary/wrapper to use for
+#           signing.
+# GPG_PATH
+#           Optional variable for specifying the gnupg "home" directory:
+#
+
+inherit sanity
+
+IPK_SIGN_PACKAGES = '1'
+IPK_GPG_BACKEND ?= 'local'
+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
+
+python () {
+    # Check configuration
+    for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
+        if not d.getVar(var, True):
+            raise_sanity_error("You need to define %s in the config" % var, d)
+
+    sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
+    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+        raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+}
+
+def sign_ipk(d, ipk_to_sign):
+    from oe.gpg_sign import get_signer
+
+    bb.debug(1, 'Signing ipk: %s' % ipk_to_sign)
+
+    signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
+    sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
+    is_ascii_sig = (sig_type.upper() != "BIN")
+
+    signer.detach_sign(ipk_to_sign,
+                       d.getVar('IPK_GPG_NAME', True),
+                       d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
+                       armor=is_ascii_sig)
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ada1b2f..ef47d1a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -1,5 +1,6 @@
 """Helper module for GPG signing"""
 import os
+import sys
 
 import bb
 import oe.utils
@@ -50,6 +51,7 @@ class LocalSigner(object):
             bb.error('rpmsign failed: %s' % proc.before.strip())
             raise bb.build.FuncFailed("Failed to sign RPM packages")
 
+
     def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
         """Create a detached signature of a file"""
         import subprocess
@@ -57,23 +59,45 @@ class LocalSigner(object):
         if passphrase_file and passphrase:
             raise Exception("You should use either passphrase_file of passphrase, not both")
 
-        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty', '--yes',
-               '-u', keyid]
-        if passphrase_file:
-            cmd += ['--passphrase-file', passphrase_file]
-        else:
-            cmd += ['--passphrase-fd', '0']
+        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty', '--yes', '-u', keyid]
+
         if self.gpg_path:
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
-        cmd.append(input_file)
-        job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-                               stderr=subprocess.PIPE)
-        _, stderr = job.communicate(passphrase)
-        if job.returncode:
-            raise bb.build.FuncFailed("Failed to create signature for '%s': %s" %
-                                      (input_file, stderr))
+
+        try:
+            keypipe = os.pipe()
+
+            if passphrase_file:
+                with open(passphrase_file) as fobj:
+                    os.write(keypipe[1], fobj.readline());
+            else:
+                os.write(keypipe[1], passphrase)
+
+            cmd += ["--passphrase-fd",  str(keypipe[0])]
+            cmd += [input_file]
+
+            job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+            (_, stderr) = job.communicate(passphrase)
+
+            os.close(keypipe[1])
+            os.close(keypipe[0])
+
+            if job.returncode:
+                raise bb.build.FuncFailed("GPG exited with code %d: %s" %
+                                          (job.returncode, stderr))
+
+        except IOError as e:
+            bb.error("IO error (%s): %s" % (e.errno, e.strerror))
+            raise Exception("Failed to sign '%s'" % input_file)
+        except OSError as e:
+            bb.error("OS error (%s): %s" % (e.errno, e.strerror))
+            raise Exception("Failed to sign '%s" % input_file)
+        except:
+            bb.error("Unexpected error (%s): %s" % (sys.exc_info()[0], sys.exc_info()[1]))
+            raise Exception("Failed to sign '%s'" % input_file)
+
 
     def verify(self, sig_file):
         """Verify signature"""
-- 
2.7.1

[PATCH v6 2/4] gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor

[Ioan-Adrian Ratiu]

Starting from v2.1 passing passwords directly to gpg does not work
anymore [1], instead a loopback interface must be used otherwise
gpg >2.1 will error out with:
"gpg: signing failed: Inappropriate ioctl for device"

gpg <2.1 does not work with the new --pinentry-mode arg and gives an
invalid option error, so we detect what is the running version of gpg
and pass it accordingly.

[1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/lib/oe/gpg_sign.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ef47d1a..9f6b0f0 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -66,6 +66,12 @@ class LocalSigner(object):
         if armor:
             cmd += ['--armor']
 
+        #gpg > 2.1 supports password pipes only through the loopback interface
+        #gpg < 2.1 errors out if given unknown parameters
+        gpg_ver = self.get_gpg_version()
+        if gpg_ver > 2.1:
+            cmd += ['--pinentry-mode', 'loopback']
+
         try:
             keypipe = os.pipe()
 
@@ -99,6 +105,20 @@ class LocalSigner(object):
             raise Exception("Failed to sign '%s'" % input_file)
 
 
+    def get_gpg_version(self):
+        """Return the gpg version"""
+        import subprocess
+
+        job = subprocess.Popen([self.gpg_bin, "--version"], stdout=subprocess.PIPE)
+        (stdout, _) = job.communicate()
+
+        if job.returncode:
+            raise bb.build.FuncFailed("Could not get gpg version (is %s installed?)" %
+                                      self.gpg_bin)
+
+        return stdout.split()[2]
+
+
     def verify(self, sig_file):
         """Verify signature"""
         cmd = self.gpg_bin + " --verify "
-- 
2.7.1

[PATCH v6 3/4] gpg_sign: export_pubkey: add signature type support

[Ioan-Adrian Ratiu]

Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/lib/oe/gpg_sign.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 9f6b0f0..9b3864a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -13,12 +13,14 @@ class LocalSigner(object):
         self.gpg_path = d.getVar('GPG_PATH', True)
         self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
 
-    def export_pubkey(self, output_file, keyid):
+    def export_pubkey(self, output_file, keyid, armor=True):
         """Export GPG public key to a file"""
-        cmd = '%s --batch --yes --export --armor -o %s ' % \
+        cmd = '%s --batch --yes --export -o %s ' % \
                 (self.gpg_bin, output_file)
         if self.gpg_path:
             cmd += "--homedir %s " % self.gpg_path
+        if armor:
+            cmd += "--armor "
         cmd += keyid
         status, output = oe.utils.getstatusoutput(cmd)
         if status:
-- 
2.7.1

[PATCH v6 4/4] package_manager: sign IPK package feeds

[Ioan-Adrian Ratiu]

Create gpg signed ipk package feeds using the gpg backend if configured.

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/classes/sign_package_feed.bbclass | 12 +++++++++++-
 meta/lib/oe/package_manager.py         | 17 +++++++++++++++--
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass
index 63ca02f..85c37f3 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -10,6 +10,12 @@
 #           Optional variable for specifying the backend to use for signing.
 #           Currently the only available option is 'local', i.e. local signing
 #           on the build host.
+# PACKAGE_FEED_GPG_SIGNATURE_TYPE
+#           Optional variable for specifying the type of gpg signature, can be:
+#               1. Ascii armored (ASC), default if not set
+#               2. Binary (BIN)
+#           This variable is only available for IPK feeds. It is ignored on
+#           other packaging backends.
 # GPG_BIN
 #           Optional variable for specifying the gpg binary/wrapper to use for
 #           signing.
@@ -20,7 +26,7 @@ inherit sanity
 
 PACKAGE_FEED_SIGN = '1'
 PACKAGE_FEED_GPG_BACKEND ?= 'local'
-
+PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC'
 
 python () {
     # Check sanity of configuration
@@ -28,6 +34,10 @@ python () {
         if not d.getVar(var, True):
             raise_sanity_error("You need to define %s in the config" % var, d)
 
+    sigtype = d.getVar("PACKAGE_FEED_GPG_SIGNATURE_TYPE", True)
+    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+        raise_sanity_error("Bad value for PACKAGE_FEED_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+
     # Set expected location of the public key
     d.setVar('PACKAGE_FEED_GPG_PUBKEY',
              os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', False),
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index b30a4da..606ba24 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -163,11 +163,16 @@ class OpkgIndexer(Indexer):
                      "MULTILIB_ARCHS"]
 
         opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
 
         if not os.path.exists(os.path.join(self.deploy_dir, "Packages")):
             open(os.path.join(self.deploy_dir, "Packages"), "w").close()
 
         index_cmds = []
+        index_sign_files = []
         for arch_var in arch_vars:
             archs = self.d.getVar(arch_var, True)
             if archs is None:
@@ -186,6 +191,8 @@ class OpkgIndexer(Indexer):
                 index_cmds.append('%s -r %s -p %s -m %s' %
                                   (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
 
+                index_sign_files.append(pkgs_file)
+
         if len(index_cmds) == 0:
             bb.note("There are no packages in %s!" % self.deploy_dir)
             return
@@ -193,9 +200,15 @@ class OpkgIndexer(Indexer):
         result = oe.utils.multiprocess_exec(index_cmds, create_index)
         if result:
             bb.fatal('%s' % ('\n'.join(result)))
-        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
-            raise NotImplementedError('Package feed signing not implementd for ipk')
 
+        if signer:
+            feed_sig_type = self.d.getVar('PACKAGE_FEED_GPG_SIGNATURE_TYPE', True)
+            is_ascii_sig = (feed_sig_type.upper() != "BIN")
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   armor=is_ascii_sig)
 
 
 class DpkgIndexer(Indexer):
-- 
2.7.1

Re: [PATCH v6 1/4] gpg_sign: add local ipk package signing functionality

[Markus Lehtonen]

Hi,

Resending as my I got a strange "Only members may post to the list."
error yesterday...

On Fri, 2016-02-19 at 17:43 +0200, Ioan-Adrian Ratiu wrote:
> Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
> module and configure signing similar to how rpm does it. sign_ipk
> uses
> gpg_sign's detach_sign because its functionality is identical to
> package
> feed signing.
> 
> IPK signing process is a bit different from rpm:
>     - Signatures are stored outside ipk files; opkg connects to a
> feed
> server and downloads them to verify a package.
>     - Signatures are of two types (both supported by opkg): binary or
> ascii armoured. By default we sign using ascii armoured.
>     - Public keys are stored on targets to verify ipks using the
> opkg-keyrings recipe.
> 
> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> ---
>  meta/classes/package_ipk.bbclass |  5 ++++
>  meta/classes/sign_ipk.bbclass    | 52
> ++++++++++++++++++++++++++++++++++++++++
>  meta/lib/oe/gpg_sign.py          | 50 ++++++++++++++++++++++++++++--
> --------
>  3 files changed, 94 insertions(+), 13 deletions(-)
>  create mode 100644 meta/classes/sign_ipk.bbclass
> 
> diff --git a/meta/classes/package_ipk.bbclass
> b/meta/classes/package_ipk.bbclass
> index 51bee28..f64837a 100644
> --- a/meta/classes/package_ipk.bbclass
> +++ b/meta/classes/package_ipk.bbclass
> @@ -246,6 +246,11 @@ python do_package_ipk () {
>              bb.utils.unlockfile(lf)
>              raise bb.build.FuncFailed("opkg-build execution failed")
>  
> +        if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
> +            ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> +            ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname,
> ipkver, d.getVar('PACKAGE_ARCH', True))
> +            sign_ipk(d, ipk_to_sign)
> +
>          cleanupcontrol(root)
>          bb.utils.unlockfile(lf)
>  
> diff --git a/meta/classes/sign_ipk.bbclass
> b/meta/classes/sign_ipk.bbclass
> new file mode 100644
> index 0000000..a481f6d
> --- /dev/null
> +++ b/meta/classes/sign_ipk.bbclass
> @@ -0,0 +1,52 @@
> +# Class for generating signed IPK packages.
> +#
> +# Configuration variables used by this class:
> +# IPK_GPG_PASSPHRASE_FILE
> +#           Path to a file containing the passphrase of the signing
> key.
> +# IPK_GPG_NAME
> +#           Name of the key to sign with.
> +# IPK_GPG_BACKEND
> +#           Optional variable for specifying the backend to use for
> signing.
> +#           Currently the only available option is 'local', i.e.
> local signing
> +#           on the build host.
> +# IPK_GPG_SIGNATURE_TYPE
> +#           Optional variable for specifying the type of gpg
> signatures, can be:
> +#                     1. Ascii armored (ASC), default if not set
> +#                     2. Binary (BIN)
> +# GPG_BIN
> +#           Optional variable for specifying the gpg binary/wrapper
> to use for
> +#           signing.
> +# GPG_PATH
> +#           Optional variable for specifying the gnupg "home"
> directory:
> +#
> +
> +inherit sanity
> +
> +IPK_SIGN_PACKAGES = '1'
> +IPK_GPG_BACKEND ?= 'local'
> +IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
> +
> +python () {
> +    # Check configuration
> +    for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
> +        if not d.getVar(var, True):
> +            raise_sanity_error("You need to define %s in the config"
> % var, d)
> +
> +    sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
> +    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
> +        raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE
> (%s), use either ASC or BIN" % sigtype)
> +}
> +
> +def sign_ipk(d, ipk_to_sign):
> +    from oe.gpg_sign import get_signer
> +
> +    bb.debug(1, 'Signing ipk: %s' % ipk_to_sign)
> +
> +    signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
> +    sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
> +    is_ascii_sig = (sig_type.upper() != "BIN")
> +
> +    signer.detach_sign(ipk_to_sign,
> +                       d.getVar('IPK_GPG_NAME', True),
> +                       d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
> +                       armor=is_ascii_sig)
> diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> index ada1b2f..ef47d1a 100644
> --- a/meta/lib/oe/gpg_sign.py
> +++ b/meta/lib/oe/gpg_sign.py
> @@ -1,5 +1,6 @@
>  """Helper module for GPG signing"""
>  import os
> +import sys
>  
>  import bb
>  import oe.utils
> @@ -50,6 +51,7 @@ class LocalSigner(object):
>              bb.error('rpmsign failed: %s' % proc.before.strip())
>              raise bb.build.FuncFailed("Failed to sign RPM packages")
>  
> +
>      def detach_sign(self, input_file, keyid, passphrase_file,
> passphrase=None, armor=True):
>          """Create a detached signature of a file"""
>          import subprocess
> @@ -57,23 +59,45 @@ class LocalSigner(object):
>          if passphrase_file and passphrase:
>              raise Exception("You should use either passphrase_file
> of passphrase, not both")
>  
> -        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty',
> '--yes',
> -               '-u', keyid]
> -        if passphrase_file:
> -            cmd += ['--passphrase-file', passphrase_file]
> -        else:
> -            cmd += ['--passphrase-fd', '0']
> +        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty',
> '--yes', '-u', keyid]
> +
>          if self.gpg_path:
>              cmd += ['--homedir', self.gpg_path]
>          if armor:
>              cmd += ['--armor']
> -        cmd.append(input_file)
> -        job = subprocess.Popen(cmd, stdin=subprocess.PIPE,
> stdout=subprocess.PIPE,
> -                               stderr=subprocess.PIPE)
> -        _, stderr = job.communicate(passphrase)
> -        if job.returncode:
> -            raise bb.build.FuncFailed("Failed to create signature
> for '%s': %s" %
> -                                      (input_file, stderr))
> +
> +        try:
> +            keypipe = os.pipe()
> +
> +            if passphrase_file:
> +                with open(passphrase_file) as fobj:
> +                    os.write(keypipe[1], fobj.readline());
> +            else:
> +                os.write(keypipe[1], passphrase)
> +
> +            cmd += ["--passphrase-fd",  str(keypipe[0])]
> +            cmd += [input_file]
> +
> +            job = subprocess.Popen(cmd, stdin=subprocess.PIPE,
> stderr=subprocess.PIPE)
> +            (_, stderr) = job.communicate(passphrase)
> +
> +            os.close(keypipe[1])
> +            os.close(keypipe[0])

I still fail to see why you want to complicate the code with os.pipe.
Why not ditch pipe and just do something like:
    if passphrase_file:
        with open(passphrase_file) as fobj:
            _, stderr = job.communicate(fobj.readline())



Thanks,
  Markus



> +
> +            if job.returncode:
> +                raise bb.build.FuncFailed("GPG exited with code %d:
> %s" %
> +                                          (job.returncode, stderr))
> +
> +        except IOError as e:
> +            bb.error("IO error (%s): %s" % (e.errno, e.strerror))
> +            raise Exception("Failed to sign '%s'" % input_file)
> +        except OSError as e:
> +            bb.error("OS error (%s): %s" % (e.errno, e.strerror))
> +            raise Exception("Failed to sign '%s" % input_file)
> +        except:
> +            bb.error("Unexpected error (%s): %s" %
> (sys.exc_info()[0], sys.exc_info()[1]))
> +            raise Exception("Failed to sign '%s'" % input_file)
> +
>  
>      def verify(self, sig_file):
>          """Verify signature"""

Re: [PATCH v6 2/4] gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor

[Markus Lehtonen]

Hi,

Resending as my I got a strange "Only members may post to the list."
error yesterday...

On Fri, 2016-02-19 at 17:43 +0200, Ioan-Adrian Ratiu wrote:
> Starting from v2.1 passing passwords directly to gpg does not work
> anymore [1], instead a loopback interface must be used otherwise
> gpg >2.1 will error out with:
> "gpg: signing failed: Inappropriate ioctl for device"
> 
> gpg <2.1 does not work with the new --pinentry-mode arg and gives an
> invalid option error, so we detect what is the running version of gpg
> and pass it accordingly.
> 
> [1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
> 
> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> ---
>  meta/lib/oe/gpg_sign.py | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> index ef47d1a..9f6b0f0 100644
> --- a/meta/lib/oe/gpg_sign.py
> +++ b/meta/lib/oe/gpg_sign.py
> @@ -66,6 +66,12 @@ class LocalSigner(object):
>          if armor:
>              cmd += ['--armor']
>  
> +        #gpg > 2.1 supports password pipes only through the loopback
> interface
> +        #gpg < 2.1 errors out if given unknown parameters
> +        gpg_ver = self.get_gpg_version()
> +        if gpg_ver > 2.1:
> +            cmd += ['--pinentry-mode', 'loopback']
> +

As far as I can tell get_gpg_version returns a string. However, you
compare that with a float. This should give more correct behavior:
+        if gpg_ver > "2.1":



Thanks,
  Markus


>          try:
>              keypipe = os.pipe()
>  
> @@ -99,6 +105,20 @@ class LocalSigner(object):
>              raise Exception("Failed to sign '%s'" % input_file)
>  
>  
> +    def get_gpg_version(self):
> +        """Return the gpg version"""
> +        import subprocess
> +
> +        job = subprocess.Popen([self.gpg_bin, "--version"],
> stdout=subprocess.PIPE)
> +        (stdout, _) = job.communicate()
> +
> +        if job.returncode:
> +            raise bb.build.FuncFailed("Could not get gpg version (is
> %s installed?)" %
> +                                      self.gpg_bin)
> +
> +        return stdout.split()[2]
> +
> +
>      def verify(self, sig_file):
>          """Verify signature"""
>          cmd = self.gpg_bin + " --verify "

Re: [PATCH v6 0/4] IPK signing for the gpg_sign module

[Markus Lehtonen]

Hi,
Resending as my I got a strange "Only members may post to the list."
error yesterday...

On Fri, 2016-02-19 at 17:45 +0200, Ioan-Adrian Ratiu wrote:
> This patch series extends the gpg_sign module to support ipk signing.
> 
> v6 implements Markus' feedback. The most notable change is the
> sign_ipk
> and detach_sign merger, as they were almost identical in
> functionality.
> This also meant a refactoring for detach_sign and a bug fix for the
> STDIN file descriptor introduced in gpg > 2.1.
> 
> Technically that STDIN bug is a feature (meh) of gpg >2.1 which
> breaks
> existing behaviour so we have to work around it i.e. check the gpg
> version and use the loopback interface. This means that gpg-agent to
> which gpg >2.1 always connects needs to be running permanently.
> 
> Ioan-Adrian Ratiu (4):
>   gpg_sign: add local ipk package signing functionality
>   gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor
>   gpg_sign: export_pubkey: add signature type support
>   package_manager: sign IPK package feeds
> 
>  meta/classes/package_ipk.bbclass       |  5 +++
>  meta/classes/sign_ipk.bbclass          | 52 ++++++++++++++++++++++++
>  meta/classes/sign_package_feed.bbclass | 12 +++++-
>  meta/lib/oe/gpg_sign.py                | 74
> +++++++++++++++++++++++++++-------
>  meta/lib/oe/package_manager.py         | 17 +++++++-
>  5 files changed, 143 insertions(+), 17 deletions(-)
>  create mode 100644 meta/classes/sign_ipk.bbclass

I just realized that it would be good to add the ipk signing key to the
signing_keys.bb recipe.

However, it'd probably be good to write it on top of Randy Witt's
rework:
http://lists.openembedded.org/pipermail/openembedded-core/2016-February
/117791.html


Thanks,
 Markus

Re: [OE-core][PATCH v6 2/4] gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor

[Burton, Ross]

On 22 February 2016 at 13:37, Markus Lehtonen <
markus.lehtonen@linux.intel.com> wrote:

> As far as I can tell get_gpg_version returns a string. However, you
> compare that with a float. This should give more correct behavior:
> +        if gpg_ver > "2.1":
>

Not sure I'd trust that either, version comparison logic is painful.
Something like this might be more resilient to interesting versioning:

dots = versionstring.split('.')
assert len(dots) >2
if int(dots[0]) >= 2 and int(dots[1])>= 1:

Also:

+        job = subprocess.Popen([self.gpg_bin, "--version"],
stdout=subprocess.PIPE)
+        (stdout, _) = job.communicate()
+
+        if job.returncode:
+            raise bb.build.FuncFailed("Could not get gpg version (is %s
installed?)" %
+                                      self.gpg_bin)
+        return stdout.split()[2]

That's a long-winded way of saying:

try:
    return subprocess.check_output((self.gpg_bin, "version")).split()[2]
except CalledProcessExceptionOrWhateverThisExceptionIsCalled:
    raise bb.build.FuncFailed("Could not get gpg version (%s). Called %s,
output %s" % (e, e.cmd, e.output))

Ross

Re: [OE-core][PATCH v6 2/4] gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor

[Burton, Ross]

On 25 February 2016 at 09:38, Markus Lehtonen <
markus.lehtonen@linux.intel.com> wrote:

> Not sure I'd trust that either, version comparison logic is painful.
> Something like this might be more resilient to interesting versioning:
>
> dots = versionstring.split('.')
> assert len(dots) >2
> if int(dots[0]) >= 2 and int(dots[1])>= 1:
>
>
> Even this doesn't work as e.g. I have gpg version 2.0.24. But, I agree
> that version comparison is tricky, e.g. "2.10" > "2.2" would return False.
> 2.1 is a nice version number in that sense ;) I don't know if gpg has had
> of will ever have any alphabets in its version number.
>

My assert should have been >=2, but surely it will work as your version
would fail the test, as intended?


> try:
>     return subprocess.check_output((self.gpg_bin, "version")).split()[2]
> except CalledProcessExceptionOrWhateverThisExceptionIsCalled:
>     raise bb.build.FuncFailed("Could not get gpg version (%s). Called %s,
> output %s" % (e, e.cmd, e.output))
>
>
> I guess you mean (with double dash before 'version'):
> return subprocess.check_output((self.gpg_bin, "--version")).split()[2]
>

Yeah, that. :)

Ross

Re: [OE-core][PATCH v6 1/4] gpg_sign: add local ipk package signing functionality

[Ioan-Adrian Ratiu]

Hello

On Mon, 22 Feb 2016 15:33:43 +0200
Markus Lehtonen <markus.lehtonen@linux.intel.com> wrote:

> On Fri, 2016-02-19 at 17:43 +0200, Ioan-Adrian Ratiu wrote:
> > Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
> > module and configure signing similar to how rpm does it. sign_ipk
> > uses
> > gpg_sign's detach_sign because its functionality is identical to
> > package
> > feed signing.
> > 
> > IPK signing process is a bit different from rpm:
> >     - Signatures are stored outside ipk files; opkg connects to a
> > feed
> > server and downloads them to verify a package.
> >     - Signatures are of two types (both supported by opkg): binary or
> > ascii armoured. By default we sign using ascii armoured.
> >     - Public keys are stored on targets to verify ipks using the
> > opkg-keyrings recipe.
> > 
> > Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > ---
> >  meta/classes/package_ipk.bbclass |  5 ++++
> >  meta/classes/sign_ipk.bbclass    | 52
> > ++++++++++++++++++++++++++++++++++++++++
> >  meta/lib/oe/gpg_sign.py          | 50 ++++++++++++++++++++++++++++--
> > --------
> >  3 files changed, 94 insertions(+), 13 deletions(-)
> >  create mode 100644 meta/classes/sign_ipk.bbclass
> > 
> > diff --git a/meta/classes/package_ipk.bbclass
> > b/meta/classes/package_ipk.bbclass
> > index 51bee28..f64837a 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -246,6 +246,11 @@ python do_package_ipk () {
> >              bb.utils.unlockfile(lf)
> >              raise bb.build.FuncFailed("opkg-build execution failed")
> >  
> > +        if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
> > +            ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> > +            ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname,
> > ipkver, d.getVar('PACKAGE_ARCH', True))
> > +            sign_ipk(d, ipk_to_sign)
> > +
> >          cleanupcontrol(root)
> >          bb.utils.unlockfile(lf)
> >  
> > diff --git a/meta/classes/sign_ipk.bbclass
> > b/meta/classes/sign_ipk.bbclass
> > new file mode 100644
> > index 0000000..a481f6d
> > --- /dev/null
> > +++ b/meta/classes/sign_ipk.bbclass
> > @@ -0,0 +1,52 @@
> > +# Class for generating signed IPK packages.
> > +#
> > +# Configuration variables used by this class:
> > +# IPK_GPG_PASSPHRASE_FILE
> > +#           Path to a file containing the passphrase of the signing
> > key.
> > +# IPK_GPG_NAME
> > +#           Name of the key to sign with.
> > +# IPK_GPG_BACKEND
> > +#           Optional variable for specifying the backend to use for
> > signing.
> > +#           Currently the only available option is 'local', i.e.
> > local signing
> > +#           on the build host.
> > +# IPK_GPG_SIGNATURE_TYPE
> > +#           Optional variable for specifying the type of gpg
> > signatures, can be:
> > +#                     1. Ascii armored (ASC), default if not set
> > +#                     2. Binary (BIN)
> > +# GPG_BIN
> > +#           Optional variable for specifying the gpg binary/wrapper
> > to use for
> > +#           signing.
> > +# GPG_PATH
> > +#           Optional variable for specifying the gnupg "home"
> > directory:
> > +#
> > +
> > +inherit sanity
> > +
> > +IPK_SIGN_PACKAGES = '1'
> > +IPK_GPG_BACKEND ?= 'local'
> > +IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
> > +
> > +python () {
> > +    # Check configuration
> > +    for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
> > +        if not d.getVar(var, True):
> > +            raise_sanity_error("You need to define %s in the config"
> > % var, d)
> > +
> > +    sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
> > +    if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
> > +        raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE
> > (%s), use either ASC or BIN" % sigtype)
> > +}
> > +
> > +def sign_ipk(d, ipk_to_sign):
> > +    from oe.gpg_sign import get_signer
> > +
> > +    bb.debug(1, 'Signing ipk: %s' % ipk_to_sign)
> > +
> > +    signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
> > +    sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
> > +    is_ascii_sig = (sig_type.upper() != "BIN")
> > +
> > +    signer.detach_sign(ipk_to_sign,
> > +                       d.getVar('IPK_GPG_NAME', True),
> > +                       d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
> > +                       armor=is_ascii_sig)
> > diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> > index ada1b2f..ef47d1a 100644
> > --- a/meta/lib/oe/gpg_sign.py
> > +++ b/meta/lib/oe/gpg_sign.py
> > @@ -1,5 +1,6 @@
> >  """Helper module for GPG signing"""
> >  import os
> > +import sys
> >  
> >  import bb
> >  import oe.utils
> > @@ -50,6 +51,7 @@ class LocalSigner(object):
> >              bb.error('rpmsign failed: %s' % proc.before.strip())
> >              raise bb.build.FuncFailed("Failed to sign RPM packages")
> >  
> > +
> >      def detach_sign(self, input_file, keyid, passphrase_file,
> > passphrase=None, armor=True):
> >          """Create a detached signature of a file"""
> >          import subprocess
> > @@ -57,23 +59,45 @@ class LocalSigner(object):
> >          if passphrase_file and passphrase:
> >              raise Exception("You should use either passphrase_file
> > of passphrase, not both")
> >  
> > -        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty',
> > '--yes',
> > -               '-u', keyid]
> > -        if passphrase_file:
> > -            cmd += ['--passphrase-file', passphrase_file]
> > -        else:
> > -            cmd += ['--passphrase-fd', '0']
> > +        cmd = [self.gpg_bin, '--detach-sign', '--batch', '--no-tty',
> > '--yes', '-u', keyid]
> > +
> >          if self.gpg_path:
> >              cmd += ['--homedir', self.gpg_path]
> >          if armor:
> >              cmd += ['--armor']
> > -        cmd.append(input_file)
> > -        job = subprocess.Popen(cmd, stdin=subprocess.PIPE,
> > stdout=subprocess.PIPE,
> > -                               stderr=subprocess.PIPE)
> > -        _, stderr = job.communicate(passphrase)
> > -        if job.returncode:
> > -            raise bb.build.FuncFailed("Failed to create signature
> > for '%s': %s" %
> > -                                      (input_file, stderr))
> > +
> > +        try:
> > +            keypipe = os.pipe()
> > +
> > +            if passphrase_file:
> > +                with open(passphrase_file) as fobj:
> > +                    os.write(keypipe[1], fobj.readline());
> > +            else:
> > +                os.write(keypipe[1], passphrase)
> > +
> > +            cmd += ["--passphrase-fd",  str(keypipe[0])]
> > +            cmd += [input_file]
> > +
> > +            job = subprocess.Popen(cmd, stdin=subprocess.PIPE,
> > stderr=subprocess.PIPE)
> > +            (_, stderr) = job.communicate(passphrase)
> > +
> > +            os.close(keypipe[1])
> > +            os.close(keypipe[0])  
> 
> I still fail to see why you want to complicate the code with os.pipe.
> Why not ditch pipe and just do something like:
>     if passphrase_file:
>         with open(passphrase_file) as fobj:
>             _, stderr = job.communicate(fobj.readline())
> 

If I don't use os.pipe, gpg >2.1 errors out:
gpg: Sorry, we are in batchmode - can't get input

Most likely it has something to do with >2.1 gpg-agent loopback
configuration but I can't seem to find its cause nor a workaround. 
Obviously removing batch mode and --no-term is not an option.

If I use os.pipe to send the passphrase, then everything works as
expected. The internet is full of issues like this, but after some
heavy googling still no fix to be found for this specific case.

Any ideas, anyone? 

Thank you,
Adrian

> 
> 
> Thanks,
>   Markus
> 
> 
> > +
> > +            if job.returncode:
> > +                raise bb.build.FuncFailed("GPG exited with code %d:
> > %s" %
> > +                                          (job.returncode, stderr))
> > +
> > +        except IOError as e:
> > +            bb.error("IO error (%s): %s" % (e.errno, e.strerror))
> > +            raise Exception("Failed to sign '%s'" % input_file)
> > +        except OSError as e:
> > +            bb.error("OS error (%s): %s" % (e.errno, e.strerror))
> > +            raise Exception("Failed to sign '%s" % input_file)
> > +        except:
> > +            bb.error("Unexpected error (%s): %s" %
> > (sys.exc_info()[0], sys.exc_info()[1]))
> > +            raise Exception("Failed to sign '%s'" % input_file)
> > +
> >  
> >      def verify(self, sig_file):
> >          """Verify signature"""  
>