From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 08/10] package: Refactor to remove isElf/is_elf function duplication
Date: Fri, 20 Jul 2018 11:39:46 +0100 [thread overview]
Message-ID: <20180720103948.30044-8-richard.purdie@linuxfoundation.org> (raw)
In-Reply-To: <20180720103948.30044-1-richard.purdie@linuxfoundation.org>
There are probably further cleanups needed here but this at least removes
the major code duplication between these two similar funcitons, keeping the
kernel module ".ko" extension check for efficiency to avoid opening and
reading file contents in the general case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/package.bbclass | 40 ++------------------
meta/lib/oe/package.py | 73 +++++++++++++++++++++---------------
2 files changed, 47 insertions(+), 66 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 74c96b9b725..222b78fe165 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -936,38 +936,6 @@ python split_and_strip_files () {
sourcefile = d.expand("${WORKDIR}/debugsources.list")
bb.utils.remove(sourcefile)
- # Return type (bits):
- # 0 - not elf
- # 1 - ELF
- # 2 - stripped
- # 4 - executable
- # 8 - shared library
- # 16 - kernel module
- def isELF(path):
- type = 0
- result = subprocess.check_output(["file", "-b", path], stderr=subprocess.STDOUT).decode("utf-8")
-
- # Not stripped
- if "ELF" in result:
- type |= 1
- if "not stripped" not in result:
- type |= 2
- if "executable" in result:
- type |= 4
- if "shared" in result:
- type |= 8
- return type
-
- def isStaticLib(path):
- if path.endswith('.a') and not os.path.islink(path):
- with open(path, 'rb') as fh:
- # The magic must include the first slash to avoid
- # matching golang static libraries
- magic = b'!<arch>\x0a/'
- start = fh.read(len(magic))
- return start == magic
- return False
-
#
# First lets figure out all of the files we may have to process ... do this only once!
#
@@ -987,7 +955,7 @@ python split_and_strip_files () {
if file.endswith(".ko") and file.find("/lib/modules/") != -1:
kernmods.append(file)
continue
- if isStaticLib(file):
+ if oe.package.is_static_lib(file):
staticlibs.append(file)
continue
@@ -1017,14 +985,14 @@ python split_and_strip_files () {
# If it's a symlink, and points to an ELF file, we capture the readlink target
if cpath.islink(file):
target = os.readlink(file)
- if isELF(ltarget):
- #bb.note("Sym: %s (%d)" % (ltarget, isELF(ltarget)))
+ if oe.package.is_elf(ltarget):
+ #bb.note("Sym: %s (%d)" % (ltarget, oe.package.is_elf(ltarget)))
symlinks[file] = target
continue
# It's a file (or hardlink), not a link
# ...but is it ELF, and is it already stripped?
- elf_file = isELF(file)
+ elf_file = oe.package.is_elf(file)
if elf_file & 1:
if elf_file & 2:
if 'already-stripped' in (d.getVar('INSANE_SKIP_' + pn) or "").split():
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 8a303106a9f..a435d661a62 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -1,8 +1,11 @@
+import mmap
+import subprocess
+
def runstrip(arg):
# Function to strip a single file, called from split_and_strip_files below
# A working 'file' (one which works on the target architecture)
#
- # The elftype is a bit pattern (explained in split_and_strip_files) to tell
+ # The elftype is a bit pattern (explained in is_elf below) to tell
# us what type of file we're processing...
# 4 - executable
# 8 - shared library
@@ -44,6 +47,44 @@ def runstrip(arg):
return
+# Detect .ko module by searching for "vermagic=" string
+def is_kernel_module(path):
+ with open(path) as f:
+ return mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ).find(b"vermagic=") >= 0
+
+# Return type (bits):
+# 0 - not elf
+# 1 - ELF
+# 2 - stripped
+# 4 - executable
+# 8 - shared library
+# 16 - kernel module
+def is_elf(path):
+ exec_type = 0
+ result = subprocess.check_output(["file", "-b", path], stderr=subprocess.STDOUT).decode("utf-8")
+
+ if "ELF" in result:
+ exec_type |= 1
+ if "not stripped" not in result:
+ exec_type |= 2
+ if "executable" in result:
+ exec_type |= 4
+ if "shared" in result:
+ exec_type |= 8
+ if "relocatable" in result:
+ if path.endswith(".ko") and path.find("/lib/modules/") != -1 and is_kernel_module(path):
+ exec_type |= 16
+ return exec_type
+
+def is_static_lib(path):
+ if path.endswith('.a') and not os.path.islink(path):
+ with open(path, 'rb') as fh:
+ # The magic must include the first slash to avoid
+ # matching golang static libraries
+ magic = b'!<arch>\x0a/'
+ start = fh.read(len(magic))
+ return start == magic
+ return False
def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, qa_already_stripped=False):
"""
@@ -56,35 +97,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, qa_already_stripped=
:param qa_already_stripped: Set to True if already-stripped' in ${INSANE_SKIP}
This is for proper logging and messages only.
"""
- import stat, errno, oe.path, oe.utils, mmap, subprocess
-
- # Detect .ko module by searching for "vermagic=" string
- def is_kernel_module(path):
- with open(path) as f:
- return mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ).find(b"vermagic=") >= 0
-
- # Return type (bits):
- # 0 - not elf
- # 1 - ELF
- # 2 - stripped
- # 4 - executable
- # 8 - shared library
- # 16 - kernel module
- def is_elf(path):
- exec_type = 0
- result = subprocess.check_output(["file", "-b", path], stderr=subprocess.STDOUT).decode("utf-8")
-
- if "ELF" in result:
- exec_type |= 1
- if "not stripped" not in result:
- exec_type |= 2
- if "executable" in result:
- exec_type |= 4
- if "shared" in result:
- exec_type |= 8
- if "relocatable" in result and is_kernel_module(path):
- exec_type |= 16
- return exec_type
+ import stat, errno, oe.path, oe.utils
elffiles = {}
inodes = {}
--
2.17.1
next prev parent reply other threads:[~2018-07-20 10:40 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-20 10:39 [PATCH 01/10] utils: Add multiprocess_launch API and testcase Richard Purdie
2018-07-20 10:39 ` [PATCH 02/10] package: Rework PACKAGELOCK based upon sstate for do_packagedata Richard Purdie
2018-07-20 10:39 ` [PATCH 03/10] package_ipk|deb: Use oe.utils.multiprocess_launch Richard Purdie
2018-07-20 10:39 ` [PATCH 04/10] package: Drop subshell usage for dwarfsrcfile generation Richard Purdie
2018-07-20 10:39 ` [PATCH 05/10] package: Don't use subshell to execute file Richard Purdie
2018-07-20 10:39 ` [PATCH 06/10] package: Allow parallel processing of debug splitting Richard Purdie
2018-07-20 10:39 ` [PATCH 07/10] package: Allow parallel processing of shlib analysis Richard Purdie
2018-07-20 10:39 ` Richard Purdie [this message]
2018-07-20 10:39 ` [PATCH 09/10] package: Call file to determine elf status in parallel Richard Purdie
2018-07-20 10:39 ` [PATCH 10/10] sstate/bitbake.conf: Use pigz if available Richard Purdie
2018-07-20 11:02 ` ✗ patchtest: failure for "utils: Add multiprocess_launch..." and 9 more Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180720103948.30044-8-richard.purdie@linuxfoundation.org \
--to=richard.purdie@linuxfoundation.org \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox