* [PATCH 1/3] python3: upgrade to 3.7.2
@ 2019-02-01 12:58 Alexander Kanavin
2019-02-01 12:58 ` [PATCH 2/3] python3: delete old 3.5.6 version Alexander Kanavin
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Alexander Kanavin @ 2019-02-01 12:58 UTC (permalink / raw)
To: openembedded-core
I took the same approach as the recent perl upgrade: write recipe from scratch,
taking the pieces from the old recipe only when they were proven to be necessary.
The pgo, manifest and ptest features are all preserved.
New features:
- native and target recipes are now unified into one recipe
- check_build_completeness.py runs right after do_compile() and verifies that
all optional modules have been built (a notorious source of regressions)
- a new approach to sysconfig.py and distutils/sysconfig.py returning values
appropriate for native or target builds: we copy the configuration file to a
separate folder, add that folder to sys.path (through environment variable
that differs between native and target builds), and point python to the file
through another environment variable.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
meta/classes/python3-dir.bbclass | 2 +-
meta/classes/python3native.bbclass | 2 +
...ib-termcap-to-linker-flags-to-avoid-.patch | 25 +
...hell-version-of-python-config-that-w.patch | 35 +
...-qemu-wrapper-when-gathering-profile.patch | 25 +
...fig-append-STAGING_LIBDIR-python-sys.patch | 42 +
...tutils-prefix-is-inside-staging-area.patch | 54 +
.../python3/avoid_warning_about_tkinter.patch | 36 +
.../python-sanity/python3/cgi_py.patch | 32 +
.../python3/check_build_completeness.py | 17 +
.../python-sanity/python3/create_manifest3.py | 433 ++++++
.../python-sanity/python3/get_module_deps3.py | 146 ++
.../python-sanity/python3/python-config.patch | 46 +
.../python3/python3-manifest.json | 1227 +++++++++++++++++
.../python-sanity/python3/run-ptest | 3 +
.../python-sanity/python3_3.7.2.bb | 281 ++++
16 files changed, 2405 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch
create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py
create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch
create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json
create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest
create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb
diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass
index 06bb046d9c2..7dd130bad99 100644
--- a/meta/classes/python3-dir.bbclass
+++ b/meta/classes/python3-dir.bbclass
@@ -1,4 +1,4 @@
-PYTHON_BASEVERSION = "3.5"
+PYTHON_BASEVERSION = "3.7"
PYTHON_ABI = "m"
PYTHON_DIR = "python${PYTHON_BASEVERSION}"
PYTHON_PN = "python3"
diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass
index da12a714703..a3acaf61bbd 100644
--- a/meta/classes/python3native.bbclass
+++ b/meta/classes/python3native.bbclass
@@ -9,6 +9,8 @@ DEPENDS_append = " python3-native "
export STAGING_INCDIR
export STAGING_LIBDIR
+export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+
# suppress host user's site-packages dirs.
export PYTHONNOUSERSITE = "1"
diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
new file mode 100644
index 00000000000..09f279ba1d7
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
@@ -0,0 +1,25 @@
+From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex.kanavin@gmail.com>
+Date: Fri, 25 Jan 2019 19:04:13 +0100
+Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host
+ contamination
+
+Upstream-Status: Inappropriate [oe-core specific]
+Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
+
+---
+ setup.py | 1 -
+ 1 file changed, 1 deletion(-)
+
+diff --git a/setup.py b/setup.py
+index b4357e3..fbec00d 100644
+--- a/setup.py
++++ b/setup.py
+@@ -856,7 +856,6 @@ class PyBuildExt(build_ext):
+ 'termcap'):
+ readline_libs.append('termcap')
+ exts.append( Extension('readline', ['readline.c'],
+- library_dirs=['/usr/lib/termcap'],
+ extra_link_args=readline_extra_link_args,
+ libraries=readline_libs) )
+ else:
diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
new file mode 100644
index 00000000000..83fd52d87f4
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
@@ -0,0 +1,35 @@
+From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex.kanavin@gmail.com>
+Date: Tue, 29 Jan 2019 15:03:01 +0100
+Subject: [PATCH] Do not use the shell version of python-config that was
+ introduced in 3.4
+
+Revert instead to the original python version: it has our tweaks and
+outputs directories correctly.
+
+Upstream-Status: Inappropriate [oe-specific]
+Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
+---
+ Makefile.pre.in | 9 +++------
+ 1 file changed, 3 insertions(+), 6 deletions(-)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 2d2e11f..cc19942 100644
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
+ sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
+ @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
+ LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
+- @ # On Darwin, always use the python version of the script, the shell
+- @ # version doesn't use the compiler customizations that are provided
+- @ # in python (_osx_support.py).
+- @if test `uname -s` = Darwin; then \
+- cp python-config.py python-config; \
+- fi
++ @ # In OpenEmbedded, always use the python version of the script, the shell
++ @ # version is broken in multiple ways, and doesn't return correct directories
++ cp python-config.py python-config
+
+
+ # Install the include files
diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
new file mode 100644
index 00000000000..fa7735ff93e
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
@@ -0,0 +1,25 @@
+From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex.kanavin@gmail.com>
+Date: Wed, 30 Jan 2019 12:41:04 +0100
+Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data
+
+Upstream-Status: Inappropriate [oe-core specific]
+Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
+---
+ Makefile.pre.in | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index a3a02a7..d5503dd 100644
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -507,8 +507,7 @@ build_all_generate_profile:
+ $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)"
+
+ run_profile_task:
+- @ # FIXME: can't run for a cross build
+- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true
+
+ build_all_merge_profile:
+ $(LLVM_PROF_MERGER)
diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
new file mode 100644
index 00000000000..2c5b76f131c
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
@@ -0,0 +1,42 @@
+From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex.kanavin@gmail.com>
+Date: Thu, 31 Jan 2019 16:46:30 +0100
+Subject: [PATCH] distutils/sysconfig: append
+ STAGING_LIBDIR/python-sysconfigdata to sys.path
+
+So that target configuration can be used when running native python
+
+Upstream-Status: Inappropriate [oe-core specific]
+Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
+
+---
+ Lib/distutils/sysconfig.py | 2 ++
+ Lib/sysconfig.py | 2 ++
+ 2 files changed, 4 insertions(+)
+
+diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
+index e07a6c8..6b8c129 100644
+--- a/Lib/distutils/sysconfig.py
++++ b/Lib/distutils/sysconfig.py
+@@ -421,6 +421,8 @@ def _init_posix():
+ platform=sys.platform,
+ multiarch=getattr(sys.implementation, '_multiarch', ''),
+ ))
++ if 'STAGING_LIBDIR' in os.environ:
++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
+ _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
+ build_time_vars = _temp.build_time_vars
+ global _config_vars
+diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
+index 9ee4d31..e586abd 100644
+--- a/Lib/sysconfig.py
++++ b/Lib/sysconfig.py
+@@ -412,6 +412,8 @@ def _init_posix(vars):
+ """Initialize the module as appropriate for POSIX systems."""
+ # _sysconfigdata is generated at build time, see _generate_posix_vars()
+ name = _get_sysconfigdata_name()
++ if 'STAGING_LIBDIR' in os.environ:
++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
+ _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
+ build_time_vars = _temp.build_time_vars
+ vars.update(build_time_vars)
diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
new file mode 100644
index 00000000000..abf08f59dd7
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
@@ -0,0 +1,54 @@
+From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Tue, 14 May 2013 15:00:26 -0700
+Subject: [PATCH] python3: Add target and native recipes
+
+Upstream-Status: Inappropriate [embedded specific]
+
+02/2015 Rebased for Python 3.4.2
+
+# The proper prefix is inside our staging area.
+# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de>
+# Signed-off-by: Phil Blundell <philb@gnu.org>
+# Signed-off-by: Khem Raj <raj.khem@gmail.com>
+# Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
+
+---
+ Lib/distutils/sysconfig.py | 10 ++++++++--
+ 1 file changed, 8 insertions(+), 2 deletions(-)
+
+diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
+index 6b8c129..3ca7f79 100644
+--- a/Lib/distutils/sysconfig.py
++++ b/Lib/distutils/sysconfig.py
+@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None):
+ If 'prefix' is supplied, use it instead of sys.base_prefix or
+ sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
+ """
+- if prefix is None:
++ if prefix is None and os.environ['STAGING_INCDIR'] != "":
++ prefix = os.environ['STAGING_INCDIR'].rstrip('include')
++ elif prefix is None:
+ prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
+ if os.name == "posix":
+ if python_build:
+@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
+ If 'prefix' is supplied, use it instead of sys.base_prefix or
+ sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
+ """
++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1]
++ if prefix is None and os.environ['STAGING_LIBDIR'] != "":
++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename)
++
+ if prefix is None:
+ if standard_lib:
+ prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
+@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
+
+ if os.name == "posix":
+ libpython = os.path.join(prefix,
+- "lib", "python" + get_python_version())
++ lib_basename, "python" + get_python_version())
+ if standard_lib:
+ return libpython
+ else:
diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
new file mode 100644
index 00000000000..24e67b4ca14
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
@@ -0,0 +1,36 @@
+From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001
+From: Andrei Gherzan <andrei@gherzan.ro>
+Date: Mon, 28 Jan 2019 15:57:54 +0000
+Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet
+ integrated in yocto so we skip the check for this module. Avoid a warning by
+ not adding this module to missing variable.
+
+Upstream-Status: Inappropriate [distribution]
+
+Also simply disable the tk module since its not in DEPENDS.
+Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
+
+---
+ setup.py | 8 +++++---
+ 1 file changed, 5 insertions(+), 3 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index fbec00d..b7a36a6 100644
+--- a/setup.py
++++ b/setup.py
+@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext):
+ self.extensions.extend(exts)
+
+ # Call the method for detecting whether _tkinter can be compiled
+- self.detect_tkinter(inc_dirs, lib_dirs)
++ # self.detect_tkinter(inc_dirs, lib_dirs)
+
+- if '_tkinter' not in [e.name for e in self.extensions]:
+- missing.append('_tkinter')
++ # tkinter module will not be avalaible as yocto
++ # doesn't have tk integrated (yet)
++ #if '_tkinter' not in [e.name for e in self.extensions]:
++ # missing.append('_tkinter')
+
+ # Build the _uuid module if possible
+ uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"])
diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch
new file mode 100644
index 00000000000..6c4ba54320b
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch
@@ -0,0 +1,32 @@
+From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001
+From: Mark Hatle <mark.hatle@windriver.com>
+Date: Wed, 21 Sep 2011 20:55:33 -0500
+Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment
+
+Upstream-Status: Inappropriate [distribution]
+
+Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
+
+---
+ Lib/cgi.py | 11 +----------
+ 1 file changed, 1 insertion(+), 10 deletions(-)
+
+diff --git a/Lib/cgi.py b/Lib/cgi.py
+index 8cf6687..094c7b4 100755
+--- a/Lib/cgi.py
++++ b/Lib/cgi.py
+@@ -1,13 +1,4 @@
+-#! /usr/local/bin/python
+-
+-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is
+-# intentionally NOT "/usr/bin/env python". On many systems
+-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI
+-# scripts, and /usr/local/bin is the default directory where Python is
+-# installed, so /usr/bin/env would be unable to find python. Granted,
+-# binary installations by Linux vendors often install Python in
+-# /usr/bin. So let those vendors patch cgi.py to match their choice
+-# of installation.
++#! /usr/bin/env python
+
+ """Support module for CGI (Common Gateway Interface) scripts.
+
diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
new file mode 100755
index 00000000000..a1eace3f571
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
@@ -0,0 +1,17 @@
+#!/usr/bin/env python3
+import sys
+logfile = open(sys.argv[1]).read()
+
+necessary_bits = logfile.find("The necessary bits to build these optional modules were not found")
+to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.")
+if necessary_bits != -1:
+ print("%s" %(logfile[necessary_bits:to_find_bits]))
+
+failed_to_build = logfile.find("Failed to build these modules:")
+if failed_to_build != -1:
+ failed_to_build_end = logfile.find("\n\n", failed_to_build)
+ print("%s" %(logfile[failed_to_build:failed_to_build_end]))
+
+if necessary_bits != -1 or failed_to_build != -1:
+ sys.exit(1)
+
diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py
new file mode 100644
index 00000000000..4da02a2991a
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py
@@ -0,0 +1,433 @@
+# This script is used as a bitbake task to create a new python manifest
+# $ bitbake python -c create_manifest
+#
+# Our goal is to keep python-core as small as posible and add other python
+# packages only when the user needs them, hence why we split upstream python
+# into several packages.
+#
+# In a very simplistic way what this does is:
+# Launch python and see specifically what is required for it to run at a minimum
+#
+# Go through the python-manifest file and launch a separate task for every single
+# one of the files on each package, this task will check what was required for that
+# specific module to run, these modules will be called dependencies.
+# The output of such task will be a list of the modules or dependencies that were
+# found for that file.
+#
+# Such output will be parsed by this script, we will look for each dependency on the
+# manifest and if we find that another package already includes it, then we will add
+# that package as an RDEPENDS to the package we are currently checking; in case we dont
+# find the current dependency on any other package we will add it to the current package
+# as part of FILES.
+#
+#
+# This way we will create a new manifest from the data structure that was built during
+# this process, on this new manifest each package will contain specifically only
+# what it needs to run.
+#
+# There are some caveats which we try to deal with, such as repeated files on different
+# packages, packages that include folders, wildcards, and special packages.
+# Its also important to note that this method only works for python files, and shared
+# libraries. Static libraries, header files and binaries need to be dealt with manually.
+#
+# This script differs from its python2 version mostly on how shared libraries are handled
+# The manifest file for python3 has an extra field which contains the cached files for
+# each package.
+# Tha method to handle cached files does not work when a module includes a folder which
+# itself contains the pycache folder, gladly this is almost never the case.
+#
+# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com>
+
+
+import sys
+import subprocess
+import json
+import os
+import collections
+
+# Get python version from ${PYTHON_MAJMIN}
+pyversion = str(sys.argv[1])
+
+# Hack to get native python search path (for folders), not fond of it but it works for now
+pivot = 'recipe-sysroot-native'
+for p in sys.path:
+ if pivot in p:
+ nativelibfolder = p[:p.find(pivot)+len(pivot)]
+
+# Empty dict to hold the whole manifest
+new_manifest = collections.OrderedDict()
+
+# Check for repeated files, folders and wildcards
+allfiles = []
+repeated = []
+wildcards = []
+
+hasfolders = []
+allfolders = []
+
+def isFolder(value):
+ value = value.replace('${PYTHON_MAJMIN}',pyversion)
+ if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
+ return True
+ else:
+ return False
+
+def isCached(item):
+ if '__pycache__' in item:
+ return True
+ else:
+ return False
+
+def prepend_comments(comments, json_manifest):
+ with open(json_manifest, 'r+') as manifest:
+ json_contents = manifest.read()
+ manifest.seek(0, 0)
+ manifest.write(comments + json_contents)
+
+# Read existing JSON manifest
+with open('python3-manifest.json') as manifest:
+ # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker
+ manifest_str = manifest.read()
+ json_start = manifest_str.find('# EOC') + 6 # EOC + \n
+ manifest.seek(0)
+ comments = manifest.read(json_start)
+ manifest_str = manifest.read()
+ old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict)
+
+#
+# First pass to get core-package functionality, because we base everything on the fact that core is actually working
+# Not exactly the same so it should not be a function
+#
+
+print ('Getting dependencies for package: core')
+
+
+# This special call gets the core dependencies and
+# appends to the old manifest so it doesnt hurt what it
+# currently holds.
+# This way when other packages check for dependencies
+# on the new core package, they will still find them
+# even when checking the old_manifest
+
+output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8')
+for coredep in output.split():
+ coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}')
+ if isCached(coredep):
+ if coredep not in old_manifest['core']['cached']:
+ old_manifest['core']['cached'].append(coredep)
+ else:
+ if coredep not in old_manifest['core']['files']:
+ old_manifest['core']['files'].append(coredep)
+
+
+# The second step is to loop through the existing files contained in the core package
+# according to the old manifest, identify if they are modules, or some other type
+# of file that we cant import (directories, binaries, configs) in which case we
+# can only assume they were added correctly (manually) so we ignore those and
+# pass them to the manifest directly.
+
+for filedep in old_manifest['core']['files']:
+ if isFolder(filedep):
+ if isCached(filedep):
+ if filedep not in old_manifest['core']['cached']:
+ old_manifest['core']['cached'].append(filedep)
+ else:
+ if filedep not in old_manifest['core']['files']:
+ old_manifest['core']['files'].append(filedep)
+ continue
+ if '${bindir}' in filedep:
+ if filedep not in old_manifest['core']['files']:
+ old_manifest['core']['files'].append(filedep)
+ continue
+ if filedep == '':
+ continue
+ if '${includedir}' in filedep:
+ if filedep not in old_manifest['core']['files']:
+ old_manifest['core']['files'].append(filedep)
+ continue
+
+ # Get actual module name , shouldnt be affected by libdir/bindir, etc.
+ pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
+
+
+ # We now know that were dealing with a python module, so we can import it
+ # and check what its dependencies are.
+ # We launch a separate task for each module for deterministic behavior.
+ # Each module will only import what is necessary for it to work in specific.
+ # The output of each task will contain each module's dependencies
+
+ print ('Getting dependencies for module: %s' % pymodule)
+ output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
+ print ('The following dependencies were found for module %s:\n' % pymodule)
+ print (output)
+
+
+ for pymodule_dep in output.split():
+ pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
+
+ if isCached(pymodule_dep):
+ if pymodule_dep not in old_manifest['core']['cached']:
+ old_manifest['core']['cached'].append(pymodule_dep)
+ else:
+ if pymodule_dep not in old_manifest['core']['files']:
+ old_manifest['core']['files'].append(pymodule_dep)
+
+
+# At this point we are done with the core package.
+# The old_manifest dictionary is updated only for the core package because
+# all others will use this a base.
+
+
+# To improve the script speed, we check which packages contain directories
+# since we will be looping through (only) those later.
+for pypkg in old_manifest:
+ for filedep in old_manifest[pypkg]['files']:
+ if isFolder(filedep):
+ print ('%s is a folder' % filedep)
+ if pypkg not in hasfolders:
+ hasfolders.append(pypkg)
+ if filedep not in allfolders:
+ allfolders.append(filedep)
+
+
+
+# This is the main loop that will handle each package.
+# It works in a similar fashion than the step before, but
+# we will now be updating a new dictionary that will eventually
+# become the new manifest.
+#
+# The following loops though all packages in the manifest,
+# through all files on each of them, and checks whether or not
+# they are modules and can be imported.
+# If they can be imported, then it checks for dependencies for
+# each of them by launching a separate task.
+# The output of that task is then parsed and the manifest is updated
+# accordingly, wether it should add the module on FILES for the current package
+# or if that module already belongs to another package then the current one
+# will RDEPEND on it
+
+for pypkg in old_manifest:
+ # Use an empty dict as data structure to hold data for each package and fill it up
+ new_manifest[pypkg] = collections.OrderedDict()
+ new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary']
+ new_manifest[pypkg]['rdepends'] = []
+ new_manifest[pypkg]['files'] = []
+ new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached']
+
+ # All packages should depend on core
+ if pypkg != 'core':
+ new_manifest[pypkg]['rdepends'].append('core')
+ new_manifest[pypkg]['cached'] = []
+
+ print('\n')
+ print('--------------------------')
+ print ('Handling package %s' % pypkg)
+ print('--------------------------')
+
+ # Handle special cases, we assume that when they were manually added
+ # to the manifest we knew what we were doing.
+ special_packages = ['misc', 'modules', 'dev', 'tests']
+ if pypkg in special_packages or 'staticdev' in pypkg:
+ print('Passing %s package directly' % pypkg)
+ new_manifest[pypkg] = old_manifest[pypkg]
+ continue
+
+ for filedep in old_manifest[pypkg]['files']:
+ # We already handled core on the first pass, we can ignore it now
+ if pypkg == 'core':
+ if filedep not in new_manifest[pypkg]['files']:
+ new_manifest[pypkg]['files'].append(filedep)
+ continue
+
+ # Handle/ignore what we cant import
+ if isFolder(filedep):
+ new_manifest[pypkg]['files'].append(filedep)
+ # Asyncio (and others) are both the package and the folder name, we should not skip those...
+ path,mod = os.path.split(filedep)
+ if mod != pypkg:
+ continue
+ if '${bindir}' in filedep:
+ if filedep not in new_manifest[pypkg]['files']:
+ new_manifest[pypkg]['files'].append(filedep)
+ continue
+ if filedep == '':
+ continue
+ if '${includedir}' in filedep:
+ if filedep not in new_manifest[pypkg]['files']:
+ new_manifest[pypkg]['files'].append(filedep)
+ continue
+
+ # Get actual module name , shouldnt be affected by libdir/bindir, etc.
+ # We need to check if the imported module comes from another (e.g. sqlite3.dump)
+ path,pymodule = os.path.split(filedep)
+ path = os.path.basename(path)
+ pymodule = os.path.splitext(os.path.basename(pymodule))[0]
+
+ # If this condition is met, it means we need to import it from another module
+ # or its the folder itself (e.g. unittest)
+ if path == pypkg:
+ if pymodule:
+ pymodule = path + '.' + pymodule
+ else:
+ pymodule = path
+
+
+
+ # We now know that were dealing with a python module, so we can import it
+ # and check what its dependencies are.
+ # We launch a separate task for each module for deterministic behavior.
+ # Each module will only import what is necessary for it to work in specific.
+ # The output of each task will contain each module's dependencies
+
+ print ('\nGetting dependencies for module: %s' % pymodule)
+ output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
+ print ('The following dependencies were found for module %s:\n' % pymodule)
+ print (output)
+
+ reportFILES = []
+ reportRDEPS = []
+
+ for pymodule_dep in output.split():
+
+ # Warning: This first part is ugly
+ # One of the dependencies that was found, could be inside of one of the folders included by another package
+ # We need to check if this happens so we can add the package containing the folder as an rdependency
+ # e.g. Folder encodings contained in codecs
+ # This would be solved if no packages included any folders
+
+ # This can be done in two ways:
+ # 1 - We assume that if we take out the filename from the path we would get
+ # the folder string, then we would check if folder string is in the list of folders
+ # This would not work if a package contains a folder which contains another folder
+ # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2
+ # folder_string would not match any value contained in the list of folders
+ #
+ # 2 - We do it the other way around, checking if the folder is contained in the path
+ # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2
+ # is folder_string inside path/folder1/folder2/filename?,
+ # Yes, it works, but we waste a couple of milliseconds.
+
+ pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
+ inFolders = False
+ for folder in allfolders:
+ # The module could have a directory named after it, e.g. xml, if we take out the filename from the path
+ # we'll end up with ${libdir}, and we want ${libdir}/xml
+ if isFolder(pymodule_dep):
+ check_path = pymodule_dep
+ else:
+ check_path = os.path.dirname(pymodule_dep)
+ if folder in check_path :
+ inFolders = True # Did we find a folder?
+ folderFound = False # Second flag to break inner for
+ # Loop only through packages which contain folders
+ for pypkg_with_folder in hasfolders:
+ if (folderFound == False):
+ # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder))
+ for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']:
+ if folder_dep == folder:
+ print ('%s folder found in %s' % (folder, pypkg_with_folder))
+ folderFound = True
+ if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg:
+ new_manifest[pypkg]['rdepends'].append(pypkg_with_folder)
+ else:
+ break
+
+ # A folder was found so we're done with this item, we can go on
+ if inFolders:
+ continue
+
+
+
+ # No directories beyond this point
+ # We might already have this module on the dictionary since it could depend on a (previously checked) module
+ if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']:
+ # Handle core as a special package, we already did it so we pass it to NEW data structure directly
+ if pypkg == 'core':
+ print('Adding %s to %s FILES' % (pymodule_dep, pypkg))
+ if pymodule_dep.endswith('*'):
+ wildcards.append(pymodule_dep)
+ if isCached(pymodule_dep):
+ new_manifest[pypkg]['cached'].append(pymodule_dep)
+ else:
+ new_manifest[pypkg]['files'].append(pymodule_dep)
+
+ # Check for repeated files
+ if pymodule_dep not in allfiles:
+ allfiles.append(pymodule_dep)
+ else:
+ if pymodule_dep not in repeated:
+ repeated.append(pymodule_dep)
+ else:
+
+
+ # Last step: Figure out if we this belongs to FILES or RDEPENDS
+ # We check if this module is already contained on another package, so we add that one
+ # as an RDEPENDS, or if its not, it means it should be contained on the current
+ # package, and we should add it to FILES
+ for possible_rdep in old_manifest:
+ # Debug
+ # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep)
+ if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']:
+ # Since were nesting, we need to check its not the same pypkg
+ if(possible_rdep != pypkg):
+ if possible_rdep not in new_manifest[pypkg]['rdepends']:
+ # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs
+ reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep))
+ new_manifest[pypkg]['rdepends'].append(possible_rdep)
+ break
+ else:
+
+ # Since this module wasnt found on another package, it is not an RDEP,
+ # so we add it to FILES for this package.
+ # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files)
+ if os.path.basename(pymodule_dep) != pypkg:
+ reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg)))
+ if isCached(pymodule_dep):
+ new_manifest[pypkg]['cached'].append(pymodule_dep)
+ else:
+ new_manifest[pypkg]['files'].append(pymodule_dep)
+ if pymodule_dep.endswith('*'):
+ wildcards.append(pymodule_dep)
+ if pymodule_dep not in allfiles:
+ allfiles.append(pymodule_dep)
+ else:
+ if pymodule_dep not in repeated:
+ repeated.append(pymodule_dep)
+
+ print('\n')
+ print('#################################')
+ print('Summary for module %s' % pymodule)
+ print('FILES found for module %s:' % pymodule)
+ print(''.join(reportFILES))
+ print('RDEPENDS found for module %s:' % pymodule)
+ print(''.join(reportRDEPS))
+ print('#################################')
+
+print('The following FILES contain wildcards, please check if they are necessary')
+print(wildcards)
+print('The following FILES contain folders, please check if they are necessary')
+print(hasfolders)
+
+
+# Sort it just so it looks nicer
+for pypkg in new_manifest:
+ new_manifest[pypkg]['files'].sort()
+ new_manifest[pypkg]['cached'].sort()
+ new_manifest[pypkg]['rdepends'].sort()
+
+# Create the manifest from the data structure that was built
+with open('python3-manifest.json.new','w') as outfile:
+ json.dump(new_manifest,outfile, indent=4)
+ outfile.write('\n')
+
+prepend_comments(comments,'python3-manifest.json.new')
+
+if (repeated):
+ error_msg = '\n\nERROR:\n'
+ error_msg += 'The following files are repeated (contained in more than one package),\n'
+ error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n'
+ error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n'
+ error_msg += '\n'.join(repeated)
+ error_msg += '\n'
+ sys.exit(error_msg)
+
diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
new file mode 100644
index 00000000000..fd12baad84e
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
@@ -0,0 +1,146 @@
+# This script is launched on separate task for each python module
+# It checks for dependencies for that specific module and prints
+# them out, the output of this execution will have all dependencies
+# for a specific module, which will be parsed an dealt on create_manifest.py
+#
+# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com>
+
+# We can get a log per module, for all the dependencies that were found, but its messy.
+debug=False
+
+import sys
+
+# We can get a list of the modules which are currently required to run python
+# so we run python-core and get its modules, we then import what we need
+# and check what modules are currently running, if we substract them from the
+# modules we had initially, we get the dependencies for the module we imported.
+
+# We use importlib to achieve this, so we also need to know what modules importlib needs
+import importlib
+
+core_deps=set(sys.modules)
+
+def fix_path(dep_path):
+ import os
+ # We DONT want the path on our HOST system
+ pivot='recipe-sysroot-native'
+ dep_path=dep_path[dep_path.find(pivot)+len(pivot):]
+
+ if '/usr/bin' in dep_path:
+ dep_path = dep_path.replace('/usr/bin''${bindir}')
+
+ # Handle multilib, is there a better way?
+ if '/usr/lib32' in dep_path:
+ dep_path = dep_path.replace('/usr/lib32','${libdir}')
+ if '/usr/lib64' in dep_path:
+ dep_path = dep_path.replace('/usr/lib64','${libdir}')
+ if '/usr/lib' in dep_path:
+ dep_path = dep_path.replace('/usr/lib','${libdir}')
+ if '/usr/include' in dep_path:
+ dep_path = dep_path.replace('/usr/include','${includedir}')
+ if '__init__.' in dep_path:
+ dep_path = os.path.split(dep_path)[0]
+ return dep_path
+
+
+# Module to import was passed as an argument
+current_module = str(sys.argv[1]).rstrip()
+if(debug==True):
+ log = open('log_%s' % current_module,'w')
+ log.write('Module %s generated the following dependencies:\n' % current_module)
+try:
+ importlib.import_module('%s' % current_module)
+except ImportError as e:
+ if (debug==True):
+ log.write('Module was not found')
+ pass
+
+
+# Get current module dependencies, dif will contain a list of specific deps for this module
+module_deps=set(sys.modules)
+
+# We handle the core package (1st pass on create_manifest.py) as a special case
+if current_module == 'python-core-package':
+ dif = core_deps
+else:
+ # We know this is not the core package, so there must be a difference.
+ dif = module_deps-core_deps
+
+
+# Check where each dependency came from
+for item in dif:
+ dep_path=''
+ try:
+ if (debug==True):
+ log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n')
+ dep_path = sys.modules['%s' % item].__file__
+ except AttributeError as e:
+ # Deals with thread (builtin module) not having __file__ attribute
+ if debug==True:
+ log.write(item + ' ')
+ log.write(str(e))
+ log.write('\n')
+ pass
+ except NameError as e:
+ # Deals with NameError: name 'dep_path' is not defined
+ # because module is not found (wasn't compiled?), e.g. bddsm
+ if (debug==True):
+ log.write(item+' ')
+ log.write(str(e))
+ pass
+
+ # Site-customize is a special case since we (OpenEmbedded) put it there manually
+ if 'sitecustomize' in dep_path:
+ dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py'
+ # Prints out result, which is what will be used by create_manifest
+ print (dep_path)
+ continue
+
+ dep_path = fix_path(dep_path)
+
+ import sysconfig
+ soabi=sysconfig.get_config_var('SOABI')
+ # Check if its a shared library and deconstruct it
+ if soabi in dep_path:
+ if (debug==True):
+ log.write('Shared library found in %s' % dep_path)
+ dep_path = dep_path.replace(soabi,'*')
+ print (dep_path)
+ continue
+
+ if (debug==True):
+ log.write(dep_path+'\n')
+ # Prints out result, which is what will be used by create_manifest
+ print (dep_path)
+
+
+ import imp
+ cpython_tag = imp.get_tag()
+ cached=''
+ # Theres no naive way to find *.pyc files on python3
+ try:
+ if (debug==True):
+ log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n')
+ cached = sys.modules['%s' % item].__cached__
+ except AttributeError as e:
+ # Deals with thread (builtin module) not having __cached__ attribute
+ if debug==True:
+ log.write(item + ' ')
+ log.write(str(e))
+ log.write('\n')
+ pass
+ except NameError as e:
+ # Deals with NameError: name 'cached' is not defined
+ if (debug==True):
+ log.write(item+' ')
+ log.write(str(e))
+ pass
+ if cached is not None:
+ if (debug==True):
+ log.write(cached)
+ cached = fix_path(cached)
+ cached = cached.replace(cpython_tag,'*')
+ print (cached)
+
+if debug==True:
+ log.close()
diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch
new file mode 100644
index 00000000000..f23b8b7df06
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch
@@ -0,0 +1,46 @@
+python-config: Revert to using distutils.sysconfig
+
+The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in
+
+12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig
+
+affect the native runtime as well as cross building. Use the old, patched
+implementation which returns paths in the staging directory and for the target,
+as appropriate.
+
+Upstream-Status: Inappropriate [Embedded Specific]
+
+Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
+:
+Index: Python-3.3.3/Misc/python-config.in
+===================================================================
+--- Python-3.3.3.orig/Misc/python-config.in
++++ Python-3.3.3/Misc/python-config.in
+@@ -4,7 +4,7 @@
+ import getopt
+ import os
+ import sys
+-import sysconfig
++from distutils import sysconfig
+
+ valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags',
+ 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir']
+@@ -32,14 +32,14 @@ if '--help' in opt_flags:
+
+ for opt in opt_flags:
+ if opt == '--prefix':
+- print(sysconfig.get_config_var('prefix'))
++ print(sysconfig.PREFIX)
+
+ elif opt == '--exec-prefix':
+- print(sysconfig.get_config_var('exec_prefix'))
++ print(sysconfig.EXEC_PREFIX)
+
+ elif opt in ('--includes', '--cflags'):
+- flags = ['-I' + sysconfig.get_path('include'),
+- '-I' + sysconfig.get_path('platinclude')]
++ flags = ['-I' + sysconfig.get_python_inc(),
++ '-I' + sysconfig.get_python_inc(plat_specific=True)]
+ if opt == '--cflags':
+ flags.extend(getvar('CFLAGS').split())
+ print(' '.join(flags))
diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
new file mode 100644
index 00000000000..82c7075f0e7
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
@@ -0,0 +1,1227 @@
+# DO NOT (entirely) modify this file manually, please read.
+#
+# IMPORTANT NOTE:
+# Please keep in mind that the create_manifest task relies on the fact the the
+# target and native Python packages are the same, and it also needs to be executed
+# with a fully working native package (with all the PACKAGECONFIGs enabled and all
+# and all the modules should be working, check log.do_compile), otherwise the script
+# will fail to find dependencies correctly, this note is valid either if you are
+# upgrading to a new Python version or adding a new package.
+#
+#
+# If you are adding a new package please follow the next steps:
+# How to add a new package:
+# - If a user wants to add a new package all that has to be done is:
+# Modify the python3-manifest.json file, and add the required file(s) to the FILES list,
+# fill up the SUMMARY section as well, the script should handle all the rest.
+#
+# Real example:
+# We want to add a web browser package, including the file webbrowser.py
+# which at the moment is on python3-misc.
+# "webbrowser": {
+# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"],
+# "rdepends": [],
+# "summary": "Python Web Browser support"}
+#
+# * Note that the rdepends field was left empty
+#
+# We run $ bitbake python3 -c create_manifest and the resulting manifest
+# should be completed after a few seconds, showing something like:
+# "webbrowser": {
+# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"],
+# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"],
+# "summary": "Python Web Browser support"}
+#
+#
+# If you are upgrading Python to a new version please follow the next steps:
+# After each Python upgrade, the create_manifest task should be executed, because we
+# don't control what changes on upstream Python, so, some module dependency
+# might have changed without us realizing it, a certain module can either have
+# more or less dependencies, or could be depending on a new file that was just
+# created on the new release and for obvious reasons we wouldn't have it on our
+# old manifest, all of these issues would cause runtime errors on our system.
+#
+# - Upgrade both the native and target Python packages to a new version
+# - Run the create_manifest task for the target Python package as its shown below:
+#
+# $ bitbake python3 -c create_manifest
+#
+# This will automatically replace your manifest file located under the Python directory
+# with an new one, which contains the new dependencies (if any).
+#
+# Several things could have gone wrong here, I will try to explain a few:
+#
+# a) A new file was introduced on this release, e.g. sha3*.so:
+# The task will check what its needed to import every module, more than one module would
+# would probably depend on sha3*.so, although only one module should contain it.
+#
+# After running the task, the new manifest will have the sha3*.so file on more than one
+# module, you need to manually decide which one of them should get it and delete it from
+# the others, for example sha3*.so should likely be on ${PN}-crypt.
+# Once you have deleted from the others you need to run the create_manifest task again,
+# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be
+# good to go.
+#
+# b) The native package wasn't built correctly and its missing a certain module:
+# As mentioned before, you need to make sure the native package was built with all the modules
+# because it is used as base to build the manifest file, you need to manually check log.do_compile
+# since it won't error out the compile function if its only missing a couple of modules.
+#
+# e.g. missing the _uuid module, log.do_compile would show the following:
+# Python build finished successfully!
+# The necessary bits to build these optional modules were not found:
+# _uuid
+#
+# What will happen here is that the new manifest would not be aware that the _uuid module exists, so
+# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on
+# the misc package (which is where any file that doesn't belong anywhere else ends up).
+#
+# This will eventually cause runtime errors on our system if we don't include the misc package on
+# on our image, because the _uuid files will be missing.
+# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be
+# detected correctly along with its dependencies, and we will get a working manifest.
+#
+# This is the reason why it is important to make sure we have a fully working native build,
+# so we can avoid these errors.
+#
+#
+#
+# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST
+# EOC
+{
+ "tests": {
+ "summary": "Python test suite",
+ "rdepends": [
+ "core",
+ "modules"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/*/test",
+ "${libdir}/python${PYTHON_MAJMIN}/*/tests",
+ "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/",
+ "${libdir}/python${PYTHON_MAJMIN}/test"
+ ],
+ "cached": []
+ },
+ "2to3": {
+ "summary": "Python automated Python 2 to 3 code translator",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${bindir}/2to3-*",
+ "${libdir}/python${PYTHON_MAJMIN}/lib2to3"
+ ],
+ "cached": []
+ },
+ "asyncio": {
+ "summary": "Python Asynchronous I/",
+ "rdepends": [
+ "core",
+ "io",
+ "logging",
+ "netclient",
+ "numbers",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/asyncio",
+ "${libdir}/python${PYTHON_MAJMIN}/concurrent",
+ "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so"
+ ],
+ "cached": []
+ },
+ "audio": {
+ "summary": "Python Audio Handling",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/chunk.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sunau.py",
+ "${libdir}/python${PYTHON_MAJMIN}/wave.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc"
+ ]
+ },
+ "codecs": {
+ "summary": "Python codec",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc"
+ ]
+ },
+ "compile": {
+ "summary": "Python bytecode compilation support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/compileall.py",
+ "${libdir}/python${PYTHON_MAJMIN}/py_compile.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc"
+ ]
+ },
+ "compression": {
+ "summary": "Python high-level compression support",
+ "rdepends": [
+ "core",
+ "shell",
+ "unixadmin"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_compression.py",
+ "${libdir}/python${PYTHON_MAJMIN}/bz2.py",
+ "${libdir}/python${PYTHON_MAJMIN}/gzip.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lzma.py",
+ "${libdir}/python${PYTHON_MAJMIN}/tarfile.py",
+ "${libdir}/python${PYTHON_MAJMIN}/zipfile.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc"
+ ]
+ },
+ "core": {
+ "summary": "Python interpreter and core modules",
+ "rdepends": [],
+ "files": [
+ "${bindir}/python*[!-config]",
+ "${includedir}/python${PYTHON_BINABI}/pyconfig*.h",
+ "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
+ "${libdir}/python${PYTHON_MAJMIN}/UserList.py",
+ "${libdir}/python${PYTHON_MAJMIN}/UserString.py",
+ "${libdir}/python${PYTHON_MAJMIN}/__future__.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py",
+ "${libdir}/python${PYTHON_MAJMIN}/abc.py",
+ "${libdir}/python${PYTHON_MAJMIN}/argparse.py",
+ "${libdir}/python${PYTHON_MAJMIN}/ast.py",
+ "${libdir}/python${PYTHON_MAJMIN}/bisect.py",
+ "${libdir}/python${PYTHON_MAJMIN}/code.py",
+ "${libdir}/python${PYTHON_MAJMIN}/codecs.py",
+ "${libdir}/python${PYTHON_MAJMIN}/codeop.py",
+ "${libdir}/python${PYTHON_MAJMIN}/collections",
+ "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py",
+ "${libdir}/python${PYTHON_MAJMIN}/configparser.py",
+ "${libdir}/python${PYTHON_MAJMIN}/contextlib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/copy.py",
+ "${libdir}/python${PYTHON_MAJMIN}/copyreg.py",
+ "${libdir}/python${PYTHON_MAJMIN}/csv.py",
+ "${libdir}/python${PYTHON_MAJMIN}/dis.py",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py",
+ "${libdir}/python${PYTHON_MAJMIN}/enum.py",
+ "${libdir}/python${PYTHON_MAJMIN}/functools.py",
+ "${libdir}/python${PYTHON_MAJMIN}/genericpath.py",
+ "${libdir}/python${PYTHON_MAJMIN}/getopt.py",
+ "${libdir}/python${PYTHON_MAJMIN}/gettext.py",
+ "${libdir}/python${PYTHON_MAJMIN}/heapq.py",
+ "${libdir}/python${PYTHON_MAJMIN}/imp.py",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py",
+ "${libdir}/python${PYTHON_MAJMIN}/inspect.py",
+ "${libdir}/python${PYTHON_MAJMIN}/io.py",
+ "${libdir}/python${PYTHON_MAJMIN}/keyword.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/linecache.py",
+ "${libdir}/python${PYTHON_MAJMIN}/locale.py",
+ "${libdir}/python${PYTHON_MAJMIN}/new.py",
+ "${libdir}/python${PYTHON_MAJMIN}/opcode.py",
+ "${libdir}/python${PYTHON_MAJMIN}/operator.py",
+ "${libdir}/python${PYTHON_MAJMIN}/optparse.py",
+ "${libdir}/python${PYTHON_MAJMIN}/os.py",
+ "${libdir}/python${PYTHON_MAJMIN}/platform.py",
+ "${libdir}/python${PYTHON_MAJMIN}/posixpath.py",
+ "${libdir}/python${PYTHON_MAJMIN}/re.py",
+ "${libdir}/python${PYTHON_MAJMIN}/reprlib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py",
+ "${libdir}/python${PYTHON_MAJMIN}/selectors.py",
+ "${libdir}/python${PYTHON_MAJMIN}/signal.py",
+ "${libdir}/python${PYTHON_MAJMIN}/site.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py",
+ "${libdir}/python${PYTHON_MAJMIN}/stat.py",
+ "${libdir}/python${PYTHON_MAJMIN}/stringprep.py",
+ "${libdir}/python${PYTHON_MAJMIN}/struct.py",
+ "${libdir}/python${PYTHON_MAJMIN}/subprocess.py",
+ "${libdir}/python${PYTHON_MAJMIN}/symbol.py",
+ "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py",
+ "${libdir}/python${PYTHON_MAJMIN}/textwrap.py",
+ "${libdir}/python${PYTHON_MAJMIN}/threading.py",
+ "${libdir}/python${PYTHON_MAJMIN}/token.py",
+ "${libdir}/python${PYTHON_MAJMIN}/tokenize.py",
+ "${libdir}/python${PYTHON_MAJMIN}/traceback.py",
+ "${libdir}/python${PYTHON_MAJMIN}/types.py",
+ "${libdir}/python${PYTHON_MAJMIN}/warnings.py",
+ "${libdir}/python${PYTHON_MAJMIN}/weakref.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__",
+ "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc"
+ ]
+ },
+ "crypt": {
+ "summary": "Python basic cryptographic and hashing support",
+ "rdepends": [
+ "core",
+ "math",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/crypt.py",
+ "${libdir}/python${PYTHON_MAJMIN}/hashlib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc"
+ ]
+ },
+ "ctypes": {
+ "summary": "Python C types support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/ctypes",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so"
+ ],
+ "cached": []
+ },
+ "curses": {
+ "summary": "Python curses support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/curses",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so"
+ ],
+ "cached": []
+ },
+ "datetime": {
+ "summary": "Python calendar and time support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_strptime.py",
+ "${libdir}/python${PYTHON_MAJMIN}/calendar.py",
+ "${libdir}/python${PYTHON_MAJMIN}/datetime.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc"
+ ]
+ },
+ "db": {
+ "summary": "Python file-based database support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/dbm",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so"
+ ],
+ "cached": []
+ },
+ "debugger": {
+ "summary": "Python debugger",
+ "rdepends": [
+ "core",
+ "pprint",
+ "shell",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/bdb.py",
+ "${libdir}/python${PYTHON_MAJMIN}/pdb.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc"
+ ]
+ },
+ "dev": {
+ "cached": [],
+ "files": [
+ "${base_libdir}/*.a",
+ "${base_libdir}/*.o",
+ "${bindir}/python*-config",
+ "${datadir}/aclocal",
+ "${datadir}/pkgconfig",
+ "${includedir}",
+ "${libdir}/*.a",
+ "${libdir}/*.la",
+ "${libdir}/*.o",
+ "${libdir}/lib*${SOLIBSDEV}",
+ "${libdir}/pkgconfig",
+ "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile",
+ "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__"
+ ],
+ "rdepends": [
+ "core"
+ ],
+ "summary": "Python development package"
+ },
+ "difflib": {
+ "summary": "Python helpers for computing deltas between objects",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/difflib.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc"
+ ]
+ },
+ "distutils-staticdev": {
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a"
+ ],
+ "rdepends": [
+ "distutils"
+ ],
+ "summary": "Python distribution utilities (static libraries)"
+ },
+ "distutils": {
+ "summary": "Python Distribution Utilities",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/distutils"
+ ],
+ "cached": []
+ },
+ "doctest": {
+ "summary": "Python framework for running examples in docstrings",
+ "rdepends": [
+ "core",
+ "debugger",
+ "difflib",
+ "logging",
+ "pprint",
+ "shell",
+ "stringold",
+ "unittest"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/doctest.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc"
+ ]
+ },
+ "email": {
+ "summary": "Python email support",
+ "rdepends": [
+ "core",
+ "crypt",
+ "datetime",
+ "io",
+ "math",
+ "netclient"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/email",
+ "${libdir}/python${PYTHON_MAJMIN}/imaplib.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc"
+ ]
+ },
+ "fcntl": {
+ "summary": "Python's fcntl interface",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so"
+ ],
+ "cached": []
+ },
+ "gdbm": {
+ "summary": "Python GNU database support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so"
+ ],
+ "cached": []
+ },
+ "html": {
+ "summary": "Python HTML processing support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/formatter.py",
+ "${libdir}/python${PYTHON_MAJMIN}/html"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc"
+ ]
+ },
+ "idle": {
+ "summary": "Python Integrated Development Environment",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${bindir}/idle*",
+ "${libdir}/python${PYTHON_MAJMIN}/idlelib"
+ ],
+ "cached": []
+ },
+ "image": {
+ "summary": "Python graphical image handling",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/colorsys.py",
+ "${libdir}/python${PYTHON_MAJMIN}/imghdr.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc"
+ ]
+ },
+ "io": {
+ "summary": "Python low-level I/O",
+ "rdepends": [
+ "compression",
+ "core",
+ "crypt",
+ "math",
+ "netclient",
+ "shell",
+ "unixadmin"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_pyio.py",
+ "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/pipes.py",
+ "${libdir}/python${PYTHON_MAJMIN}/socket.py",
+ "${libdir}/python${PYTHON_MAJMIN}/ssl.py",
+ "${libdir}/python${PYTHON_MAJMIN}/tempfile.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc"
+ ]
+ },
+ "json": {
+ "summary": "Python JSON support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/json",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so"
+ ],
+ "cached": []
+ },
+ "logging": {
+ "summary": "Python logging support",
+ "rdepends": [
+ "core",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/logging"
+ ],
+ "cached": []
+ },
+ "mailbox": {
+ "summary": "Python mailbox format support",
+ "rdepends": [
+ "core",
+ "crypt",
+ "datetime",
+ "email",
+ "fcntl",
+ "io",
+ "math",
+ "mime",
+ "netclient",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/mailbox.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc"
+ ]
+ },
+ "math": {
+ "summary": "Python math support",
+ "rdepends": [
+ "core",
+ "crypt"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/random.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc"
+ ]
+ },
+ "mime": {
+ "summary": "Python MIME handling APIs",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/quopri.py",
+ "${libdir}/python${PYTHON_MAJMIN}/uu.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc"
+ ]
+ },
+ "mmap": {
+ "summary": "Python memory-mapped file support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so"
+ ],
+ "cached": []
+ },
+ "modules": {
+ "cached": [],
+ "files": [],
+ "rdepends": [
+ "2to3",
+ "asyncio",
+ "audio",
+ "codecs",
+ "compile",
+ "compression",
+ "core",
+ "crypt",
+ "ctypes",
+ "curses",
+ "datetime",
+ "db",
+ "debugger",
+ "difflib",
+ "distutils",
+ "doctest",
+ "email",
+ "fcntl",
+ "html",
+ "idle",
+ "image",
+ "io",
+ "json",
+ "logging",
+ "mailbox",
+ "math",
+ "mime",
+ "mmap",
+ "multiprocessing",
+ "netclient",
+ "netserver",
+ "numbers",
+ "pickle",
+ "pkgutil",
+ "plistlib",
+ "pprint",
+ "profile",
+ "pydoc",
+ "resource",
+ "runpy",
+ "shell",
+ "smtpd",
+ "sqlite3",
+ "stringold",
+ "syslog",
+ "terminal",
+ "threading",
+ "tkinter",
+ "typing",
+ "unittest",
+ "unixadmin",
+ "venv",
+ "xml",
+ "xmlrpc"
+ ],
+ "summary": "All Python modules"
+ },
+ "multiprocessing": {
+ "summary": "Python multiprocessing support",
+ "rdepends": [
+ "core",
+ "io",
+ "pickle"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/multiprocessing"
+ ],
+ "cached": []
+ },
+ "netclient": {
+ "summary": "Python Internet Protocol clients",
+ "rdepends": [
+ "core",
+ "crypt",
+ "datetime",
+ "email",
+ "io",
+ "math",
+ "mime",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/base64.py",
+ "${libdir}/python${PYTHON_MAJMIN}/ftplib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/hmac.py",
+ "${libdir}/python${PYTHON_MAJMIN}/http",
+ "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py",
+ "${libdir}/python${PYTHON_MAJMIN}/nntplib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/poplib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/smtplib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py",
+ "${libdir}/python${PYTHON_MAJMIN}/urllib",
+ "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__",
+ "${libdir}/python${PYTHON_MAJMIN}/uuid.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc"
+ ]
+ },
+ "netserver": {
+ "summary": "Python Internet Protocol servers",
+ "rdepends": [
+ "compression",
+ "core",
+ "crypt",
+ "datetime",
+ "email",
+ "html",
+ "io",
+ "math",
+ "mime",
+ "netclient",
+ "shell",
+ "stringold",
+ "unixadmin"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/cgi.py",
+ "${libdir}/python${PYTHON_MAJMIN}/socketserver.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc"
+ ]
+ },
+ "numbers": {
+ "summary": "Python number APIs",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py",
+ "${libdir}/python${PYTHON_MAJMIN}/contextvars.py",
+ "${libdir}/python${PYTHON_MAJMIN}/decimal.py",
+ "${libdir}/python${PYTHON_MAJMIN}/fractions.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/numbers.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc"
+ ]
+ },
+ "pickle": {
+ "summary": "Python serialisation/persistence support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/pickle.py",
+ "${libdir}/python${PYTHON_MAJMIN}/pickletools.py",
+ "${libdir}/python${PYTHON_MAJMIN}/shelve.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc"
+ ]
+ },
+ "pkgutil": {
+ "summary": "Python package extension utility support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc"
+ ]
+ },
+ "plistlib": {
+ "summary": "Generate and parse Mac OS X .plist files",
+ "rdepends": [
+ "core",
+ "datetime",
+ "xml"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/plistlib.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc"
+ ]
+ },
+ "pprint": {
+ "summary": "Python pretty-print support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/pprint.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc"
+ ]
+ },
+ "profile": {
+ "summary": "Python basic performance profiling support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/cProfile.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/profile.py",
+ "${libdir}/python${PYTHON_MAJMIN}/pstats.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc"
+ ]
+ },
+ "pydoc": {
+ "summary": "Python interactive help support",
+ "rdepends": [
+ "core",
+ "netclient",
+ "pkgutil"
+ ],
+ "files": [
+ "${bindir}/pydoc*",
+ "${libdir}/python${PYTHON_MAJMIN}/pydoc.py",
+ "${libdir}/python${PYTHON_MAJMIN}/pydoc_data"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc"
+ ]
+ },
+ "resource": {
+ "summary": "Python resource control interface",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so"
+ ],
+ "cached": []
+ },
+ "runpy": {
+ "summary": "Python helper for locating/executing scripts in module namespace",
+ "rdepends": [
+ "core",
+ "pkgutil"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/runpy.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc"
+ ]
+ },
+ "shell": {
+ "summary": "Python shell-like functionality",
+ "rdepends": [
+ "compression",
+ "core",
+ "stringold",
+ "unixadmin"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/cmd.py",
+ "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py",
+ "${libdir}/python${PYTHON_MAJMIN}/glob.py",
+ "${libdir}/python${PYTHON_MAJMIN}/shlex.py",
+ "${libdir}/python${PYTHON_MAJMIN}/shutil.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc"
+ ]
+ },
+ "smtpd": {
+ "summary": "Python Simple Mail Transport Daemon",
+ "rdepends": [
+ "core",
+ "crypt",
+ "datetime",
+ "email",
+ "io",
+ "math",
+ "mime",
+ "netclient",
+ "stringold"
+ ],
+ "files": [
+ "${bindir}/smtpd.py",
+ "${libdir}/python${PYTHON_MAJMIN}/asynchat.py",
+ "${libdir}/python${PYTHON_MAJMIN}/asyncore.py",
+ "${libdir}/python${PYTHON_MAJMIN}/smtpd.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc"
+ ]
+ },
+ "sqlite3": {
+ "summary": "Python Sqlite3 database support",
+ "rdepends": [
+ "core",
+ "datetime"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/sqlite3"
+ ],
+ "cached": []
+ },
+ "stringold": {
+ "summary": "Python string APIs [deprecated]",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/string.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc"
+ ]
+ },
+ "syslog": {
+ "summary": "Python syslog interface",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so"
+ ],
+ "cached": []
+ },
+ "terminal": {
+ "summary": "Python terminal controlling support",
+ "rdepends": [
+ "core",
+ "io"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/pty.py",
+ "${libdir}/python${PYTHON_MAJMIN}/tty.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc"
+ ]
+ },
+ "threading": {
+ "summary": "Python threading & synchronization support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py",
+ "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/queue.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc",
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc"
+ ]
+ },
+ "tkinter": {
+ "summary": "Python Tcl/Tk bindings",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/tkinter"
+ ],
+ "cached": []
+ },
+ "typing": {
+ "summary": "Python typing support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/typing.py"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc"
+ ]
+ },
+ "unittest": {
+ "summary": "Python unit testing framework",
+ "rdepends": [
+ "core",
+ "difflib",
+ "logging",
+ "pprint",
+ "shell",
+ "stringold"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/unittest",
+ "${libdir}/python${PYTHON_MAJMIN}/unittest/",
+ "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__"
+ ],
+ "cached": []
+ },
+ "unixadmin": {
+ "summary": "Python Unix administration support",
+ "rdepends": [
+ "core",
+ "io"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/getpass.py",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so"
+ ],
+ "cached": [
+ "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc"
+ ]
+ },
+ "venv": {
+ "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.",
+ "rdepends": [
+ "compression",
+ "core",
+ "logging",
+ "shell",
+ "stringold",
+ "unixadmin"
+ ],
+ "files": [
+ "${bindir}/pyvenv*",
+ "${libdir}/python${PYTHON_MAJMIN}/venv"
+ ],
+ "cached": []
+ },
+ "xml": {
+ "summary": "Python basic XML support",
+ "rdepends": [
+ "core"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so",
+ "${libdir}/python${PYTHON_MAJMIN}/xml"
+ ],
+ "cached": []
+ },
+ "xmlrpc": {
+ "summary": "Python XML-RPC support",
+ "rdepends": [
+ "core",
+ "xml"
+ ],
+ "files": [
+ "${libdir}/python${PYTHON_MAJMIN}/xmlrpc",
+ "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__"
+ ],
+ "cached": []
+ }
+}
diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest
new file mode 100644
index 00000000000..3863c6d314f
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g'
diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
new file mode 100644
index 00000000000..61fa12ffe8d
--- /dev/null
+++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
@@ -0,0 +1,281 @@
+SUMMARY = "The Python Programming Language"
+HOMEPAGE = "http://www.python.org"
+LICENSE = "PSFv2"
+SECTION = "devel/python"
+
+LIC_FILES_CHKSUM = "file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754"
+
+SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
+ file://run-ptest \
+ file://create_manifest3.py \
+ file://get_module_deps3.py \
+ file://python3-manifest.json \
+ file://check_build_completeness.py \
+ file://cgi_py.patch \
+ file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \
+ ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \
+ file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \
+ file://python-config.patch \
+ file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \
+ "
+
+SRC_URI_append_class-native = " \
+ file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
+ file://12-distutils-prefix-is-inside-staging-area.patch \
+ "
+
+SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb"
+SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb"
+
+# exclude pre-releases for both python 2.x and 3.x
+UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar"
+
+CVE_PRODUCT = "python"
+
+PYTHON_MAJMIN = "3.7"
+PYTHON_BINABI = "${PYTHON_MAJMIN}m"
+
+S = "${WORKDIR}/Python-${PV}"
+
+BBCLASSEXTEND = "native nativesdk"
+
+inherit autotools pkgconfig qemu ptest multilib_header update-alternatives
+
+MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}"
+
+ALTERNATIVE_${PN}-dev = "python-config"
+ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config"
+ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}"
+
+
+DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2"
+DEPENDS_append_class-target = " python3-native"
+DEPENDS_append_class-nativesdk = " python3-native"
+
+EXTRA_OECONF = " --without-ensurepip --enable-shared"
+EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}"
+
+
+EXTRANATIVEPATH += "python3-native"
+
+CACHED_CONFIGUREVARS = " \
+ ac_cv_file__dev_ptmx=yes \
+ ac_cv_file__dev_ptc=no \
+"
+
+PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}"
+PACKAGECONFIG_class-native ??= "readline"
+PACKAGECONFIG_class-nativesdk ??= "readline"
+PACKAGECONFIG[readline] = ",,readline"
+# Use profile guided optimisation by running PyBench inside qemu-user
+PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native"
+PACKAGECONFIG[tk] = ",,tk"
+
+CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid"
+
+EXTRA_OEMAKE = '\
+ STAGING_LIBDIR=${STAGING_LIBDIR} \
+ STAGING_INCDIR=${STAGING_INCDIR} \
+'
+
+do_compile_prepend_class-target() {
+ if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then
+ qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}"
+ cat >pgo-wrapper <<EOF
+#!/bin/sh
+cd ${B}
+$qemu_binary "\$@"
+EOF
+ chmod +x pgo-wrapper
+ fi
+}
+
+do_install_prepend() {
+ ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile
+}
+
+do_install_append_class-target() {
+ oe_multilib_header python${PYTHON_BINABI}/pyconfig.h
+}
+
+do_install_append_class-native() {
+ # Make sure we use /usr/bin/env python
+ for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
+ sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
+ done
+ # Add a symlink to the native Python so that scripts can just invoke
+ # "nativepython" and get the right one without needing absolute paths
+ # (these often end up too long for the #! parser in the kernel as the
+ # buffer is 128 bytes long).
+ ln -s python3-native/python3 ${D}${bindir}/nativepython3
+}
+
+do_install_append() {
+ mkdir -p ${D}${libdir}/python-sysconfigdata
+ sysconfigfile=`find ${D} -name _sysconfig*.py`
+ cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
+
+ sed -i \
+ -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \
+ -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
+ -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
+ ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
+}
+
+SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py"
+PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
+
+py_package_preprocess () {
+ # Remove references to buildmachine paths in target Makefile and _sysconfigdata
+ sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \
+ -e 's|${DEBUG_PREFIX_MAP}||g' \
+ -e 's:${HOSTTOOLS_DIR}/::g' \
+ -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
+ -e 's:${RECIPE_SYSROOT}::g' \
+ -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
+ ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \
+ ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \
+ ${PKGD}/${bindir}/python${PYTHON_BINABI}-config
+
+ # Recompile _sysconfigdata after modifying it
+ cd ${PKGD}
+ sysconfigfile=`find . -name _sysconfigdata_*.py`
+ ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
+ -c "from py_compile import compile; compile('$sysconfigfile')"
+ ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
+ -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)"
+ ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
+ -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)"
+ cd -
+
+ mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}
+
+ #Remove the unneeded copy of target sysconfig data
+ rm -rf ${PKGD}/${libdir}/python-sysconfigdata
+}
+
+# We want bytecode precompiled .py files (.pyc's) by default
+# but the user may set it on their own conf
+INCLUDE_PYCS ?= "1"
+
+python(){
+ import collections, json
+
+ filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json')
+ # This python changes the datastore based on the contents of a file, so mark
+ # that dependency.
+ bb.parse.mark_dependency(d, filename)
+
+ with open(filename) as manifest_file:
+ manifest_str = manifest_file.read()
+ json_start = manifest_str.find('# EOC') + 6
+ manifest_file.seek(json_start)
+ manifest_str = manifest_file.read()
+ python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict)
+
+ # First set RPROVIDES for -native case
+ # Hardcoded since it cant be python3-native-foo, should be python3-foo-native
+ pn = 'python3'
+ rprovides = d.getVar('RPROVIDES').split()
+
+ for key in python_manifest:
+ pypackage = pn + '-' + key + '-native'
+ if pypackage not in rprovides:
+ rprovides.append(pypackage)
+
+ d.setVar('RPROVIDES_class-native', ' '.join(rprovides))
+
+ # Then work on the target
+ include_pycs = d.getVar('INCLUDE_PYCS')
+
+ packages = d.getVar('PACKAGES').split()
+ pn = d.getVar('PN')
+
+ newpackages=[]
+ for key in python_manifest:
+ pypackage= pn + '-' + key
+
+ if pypackage not in packages:
+ # We need to prepend, otherwise python-misc gets everything
+ # so we use a new variable
+ newpackages.append(pypackage)
+
+ # "Build" python's manifest FILES, RDEPENDS and SUMMARY
+ d.setVar('FILES_' + pypackage, '')
+ for value in python_manifest[key]['files']:
+ d.appendVar('FILES_' + pypackage, ' ' + value)
+
+ # Add cached files
+ if include_pycs == '1':
+ for value in python_manifest[key]['cached']:
+ d.appendVar('FILES_' + pypackage, ' ' + value)
+
+ for value in python_manifest[key]['rdepends']:
+ # Make it work with or without $PN
+ if '${PN}' in value:
+ value=value.split('-')[1]
+ d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value)
+ d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary'])
+
+ # Prepending so to avoid python-misc getting everything
+ packages = newpackages + packages
+ d.setVar('PACKAGES', ' '.join(packages))
+ d.setVar('ALLOW_EMPTY_${PN}-modules', '1')
+}
+
+# Files needed to create a new manifest
+
+do_create_manifest() {
+ # This task should be run with every new release of Python.
+ # We must ensure that PACKAGECONFIG enables everything when creating
+ # a new manifest, this is to base our new manifest on a complete
+ # native python build, containing all dependencies, otherwise the task
+ # wont be able to find the required files.
+ # e.g. BerkeleyDB is an optional build dependency so it may or may not
+ # be present, we must ensure it is.
+
+ cd ${WORKDIR}
+ # This needs to be executed by python-native and NOT by HOST's python
+ nativepython3 create_manifest3.py ${PYTHON_MAJMIN}
+ cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json
+}
+
+# bitbake python -c create_manifest
+addtask do_create_manifest
+
+# Make sure we have native python ready when we create a new manifest
+do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot"
+do_create_manifest[depends] += "python3:do_patch"
+
+# manual dependency additions
+RPROVIDES_${PN}-modules = "${PN}"
+RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules"
+RRECOMMENDS_${PN}-crypt = "openssl ca-certificates"
+
+FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}"
+FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
+FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}"
+
+# provide python-pyvenv from python3-venv
+RPROVIDES_${PN}-venv += "python3-pyvenv"
+
+# package libpython3
+PACKAGES =+ "libpython3 libpython3-staticdev"
+FILES_libpython3 = "${libdir}/libpython*.so.*"
+FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a"
+INSANE_SKIP_${PN}-dev += "dev-elf"
+
+# catch all the rest (unsorted)
+PACKAGES += "${PN}-misc"
+RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs"
+RDEPENDS_${PN}-modules += "python3-misc"
+FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}"
+
+# catch manpage
+PACKAGES += "${PN}-man"
+FILES_${PN}-man = "${datadir}/man"
+
+RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2"
+RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}"
+RDEPENDS_${PN}-dev = ""
+
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 2/3] python3: delete old 3.5.6 version 2019-02-01 12:58 [PATCH 1/3] python3: upgrade to 3.7.2 Alexander Kanavin @ 2019-02-01 12:58 ` Alexander Kanavin 2019-02-01 12:58 ` [PATCH 3/3] lib/oeqa/runtime/cases/gi.py: fix deprecation warning Alexander Kanavin 2019-02-01 20:49 ` [PATCH 1/3] python3: upgrade to 3.7.2 Khem Raj 2 siblings, 0 replies; 8+ messages in thread From: Alexander Kanavin @ 2019-02-01 12:58 UTC (permalink / raw) To: openembedded-core Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> --- .../python/python3-native_3.5.6.bb | 104 -- meta/recipes-devtools/python/python3.inc | 43 - ...hell-version-of-python-config-that-w.patch | 38 - ..._sysconfigdata.py-to-initialize-dist.patch | 66 - ...ontext-has-improved-default-settings.patch | 272 ---- ...3032-where-it-fails-with-UnicodeDeco.patch | 37 - ...asename-to-replace-CC-for-checking-c.patch | 140 -- ...d-target-to-split-profile-generation.patch | 40 - ...S-1.3-cipher-suites-and-OP_NO_TLSv1_.patch | 227 --- ...for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch | 173 --- ....3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch | 110 -- ...ALPN-changes-for-OpenSSL-1.1.0f-2305.patch | 68 - .../python3/03-fix-tkinter-detection.patch | 42 - .../python3/030-fixup-include-dirs.patch | 33 - .../080-distutils-dont_adjust_files.patch | 92 -- ...tutils-prefix-is-inside-staging-area.patch | 57 - .../python/python3/130-readline-setup.patch | 55 - .../python/python3/150-fix-setupterm.patch | 17 - ...GS-for-extensions-when-cross-compili.patch | 56 - .../python3/avoid-ncursesw-include-path.patch | 21 - .../python3/avoid_warning_about_tkinter.patch | 29 - .../python/python3/cgi_py.patch | 23 - .../python3/configure.ac-fix-LIBPL.patch | 30 - .../python/python3/create_manifest3.py | 433 ------ .../fix_for_using_different_libdir.patch | 54 - .../python/python3/float-endian.patch | 212 --- .../python/python3/ftplib.patch | 60 - .../python/python3/get_module_deps3.py | 146 -- .../python3/host_include_contamination.patch | 28 - ...ssing-libraries-to-Extension-for-mul.patch | 82 -- .../python/python3/python-3.3-multilib.patch | 363 ----- .../python/python3/python-config.patch | 46 - .../python/python3/python3-manifest.json | 1242 ----------------- ...CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch | 25 - .../python/python3/regen-all.patch | 25 - .../recipes-devtools/python/python3/run-ptest | 3 - ...check-cross_compiling-when-get-FLAGS.patch | 50 - .../python/python3/setuptweaks.patch | 57 - .../python/python3/signal.patch | 56 - .../python/python3/sitecustomize.py | 37 - ...port_SOURCE_DATE_EPOCH_in_py_compile.patch | 97 -- ...sysconfig.py-add-_PYTHON_PROJECT_SRC.patch | 51 - .../python3/sysroot-include-headers.patch | 35 - ...k-MULTIARCH-for-powerpc-linux-gnuspe.patch | 52 - ...eak-MULTIARCH-for-powerpc-linux-musl.patch | 40 - .../python/python3/unixccompiler.patch | 35 - meta/recipes-devtools/python/python3_3.5.6.bb | 334 ----- 47 files changed, 5336 deletions(-) delete mode 100644 meta/recipes-devtools/python/python3-native_3.5.6.bb delete mode 100644 meta/recipes-devtools/python/python3.inc delete mode 100644 meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch delete mode 100644 meta/recipes-devtools/python/python3/0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch delete mode 100644 meta/recipes-devtools/python/python3/0001-Issue-28043-SSLContext-has-improved-default-settings.patch delete mode 100644 meta/recipes-devtools/python/python3/0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch delete mode 100644 meta/recipes-devtools/python/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch delete mode 100644 meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch delete mode 100644 meta/recipes-devtools/python/python3/0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch delete mode 100644 meta/recipes-devtools/python/python3/0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch delete mode 100644 meta/recipes-devtools/python/python3/0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch delete mode 100644 meta/recipes-devtools/python/python3/0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch delete mode 100644 meta/recipes-devtools/python/python3/03-fix-tkinter-detection.patch delete mode 100644 meta/recipes-devtools/python/python3/030-fixup-include-dirs.patch delete mode 100644 meta/recipes-devtools/python/python3/080-distutils-dont_adjust_files.patch delete mode 100644 meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch delete mode 100644 meta/recipes-devtools/python/python3/130-readline-setup.patch delete mode 100644 meta/recipes-devtools/python/python3/150-fix-setupterm.patch delete mode 100644 meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch delete mode 100644 meta/recipes-devtools/python/python3/avoid-ncursesw-include-path.patch delete mode 100644 meta/recipes-devtools/python/python3/avoid_warning_about_tkinter.patch delete mode 100644 meta/recipes-devtools/python/python3/cgi_py.patch delete mode 100644 meta/recipes-devtools/python/python3/configure.ac-fix-LIBPL.patch delete mode 100644 meta/recipes-devtools/python/python3/create_manifest3.py delete mode 100644 meta/recipes-devtools/python/python3/fix_for_using_different_libdir.patch delete mode 100644 meta/recipes-devtools/python/python3/float-endian.patch delete mode 100644 meta/recipes-devtools/python/python3/ftplib.patch delete mode 100644 meta/recipes-devtools/python/python3/get_module_deps3.py delete mode 100644 meta/recipes-devtools/python/python3/host_include_contamination.patch delete mode 100644 meta/recipes-devtools/python/python3/pass-missing-libraries-to-Extension-for-mul.patch delete mode 100644 meta/recipes-devtools/python/python3/python-3.3-multilib.patch delete mode 100644 meta/recipes-devtools/python/python3/python-config.patch delete mode 100644 meta/recipes-devtools/python/python3/python3-manifest.json delete mode 100644 meta/recipes-devtools/python/python3/python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch delete mode 100644 meta/recipes-devtools/python/python3/regen-all.patch delete mode 100644 meta/recipes-devtools/python/python3/run-ptest delete mode 100644 meta/recipes-devtools/python/python3/setup.py-check-cross_compiling-when-get-FLAGS.patch delete mode 100644 meta/recipes-devtools/python/python3/setuptweaks.patch delete mode 100644 meta/recipes-devtools/python/python3/signal.patch delete mode 100644 meta/recipes-devtools/python/python3/sitecustomize.py delete mode 100644 meta/recipes-devtools/python/python3/support_SOURCE_DATE_EPOCH_in_py_compile.patch delete mode 100644 meta/recipes-devtools/python/python3/sysconfig.py-add-_PYTHON_PROJECT_SRC.patch delete mode 100644 meta/recipes-devtools/python/python3/sysroot-include-headers.patch delete mode 100644 meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch delete mode 100644 meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-musl.patch delete mode 100644 meta/recipes-devtools/python/python3/unixccompiler.patch delete mode 100644 meta/recipes-devtools/python/python3_3.5.6.bb diff --git a/meta/recipes-devtools/python/python3-native_3.5.6.bb b/meta/recipes-devtools/python/python3-native_3.5.6.bb deleted file mode 100644 index c41ee8bbd9e..00000000000 --- a/meta/recipes-devtools/python/python3-native_3.5.6.bb +++ /dev/null @@ -1,104 +0,0 @@ -require recipes-devtools/python/python3.inc - -DISTRO_SRC_URI ?= "file://sitecustomize.py" -DISTRO_SRC_URI_linuxstdbase = "" -SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ -file://12-distutils-prefix-is-inside-staging-area.patch \ -file://python-config.patch \ -file://030-fixup-include-dirs.patch \ -file://080-distutils-dont_adjust_files.patch \ -file://130-readline-setup.patch \ -file://150-fix-setupterm.patch \ -file://python-3.3-multilib.patch \ -file://03-fix-tkinter-detection.patch \ -${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \ -file://0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch \ -file://sysroot-include-headers.patch \ -file://unixccompiler.patch \ -${DISTRO_SRC_URI} \ -file://sysconfig.py-add-_PYTHON_PROJECT_SRC.patch \ -file://setup.py-check-cross_compiling-when-get-FLAGS.patch \ -file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \ -file://support_SOURCE_DATE_EPOCH_in_py_compile.patch \ -file://regen-all.patch \ -file://0001-Issue-28043-SSLContext-has-improved-default-settings.patch \ -file://0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch \ -file://0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch \ -file://0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch \ -file://0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch \ -" -PACKAGECONFIG[tk] = ",,tk-native" - -EXTRANATIVEPATH += "bzip2-native" -DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native sqlite3-native gdbm-native" - -inherit native - -EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip" - -EXTRA_OEMAKE = '\ - LIBC="" \ - STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE} \ - STAGING_INCDIR=${STAGING_INCDIR_NATIVE} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ -' - -do_configure_append() { - autoreconf --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi - sed -i -e 's,#define HAVE_GETRANDOM 1,/\* #undef HAVE_GETRANDOM \*/,' ${B}/pyconfig.h -} - -# Regenerate all of the generated files -# This ensures that pgen and friends get created during the compile phase -# -do_compile_prepend() { - # Assuming https://bugs.python.org/issue33080 has been addressed in Makefile. - oe_runmake regen-all -} - -do_install() { - install -d ${D}${libdir}/pkgconfig - oe_runmake 'DESTDIR=${D}' install - if [ -e ${WORKDIR}/sitecustomize.py ]; then - install -m 0644 ${WORKDIR}/sitecustomize.py ${D}/${libdir}/python${PYTHON_MAJMIN} - fi - install -d ${D}${bindir}/${PN} - install -m 0755 Parser/pgen ${D}${bindir}/${PN} - - # Make sure we use /usr/bin/env python - for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do - sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT - done - - # Add a symlink to the native Python so that scripts can just invoke - # "nativepython" and get the right one without needing absolute paths - # (these often end up too long for the #! parser in the kernel as the - # buffer is 128 bytes long). - ln -s python3-native/python3 ${D}${bindir}/nativepython3 -} - -python(){ - - # Read JSON manifest - import json - pythondir = d.getVar('THISDIR') - with open(pythondir+'/python3/python3-manifest.json') as manifest_file: - manifest_str = manifest_file.read() - json_start = manifest_str.find('# EOC') + 6 - manifest_file.seek(json_start) - manifest_str = manifest_file.read() - python_manifest = json.loads(manifest_str) - - rprovides = d.getVar('RPROVIDES').split() - - # Hardcoded since it cant be python3-native-foo, should be python3-foo-native - pn = 'python3' - - for key in python_manifest: - pypackage = pn + '-' + key + '-native' - if pypackage not in rprovides: - rprovides.append(pypackage) - - d.setVar('RPROVIDES', ' '.join(rprovides)) -} diff --git a/meta/recipes-devtools/python/python3.inc b/meta/recipes-devtools/python/python3.inc deleted file mode 100644 index f565b3f1714..00000000000 --- a/meta/recipes-devtools/python/python3.inc +++ /dev/null @@ -1,43 +0,0 @@ -SUMMARY = "The Python Programming Language" -HOMEPAGE = "http://www.python.org" -LICENSE = "PSFv2" -SECTION = "devel/python" - -# TODO Remove this when we upgrade -INC_PR = "r1" -PR = "${INC_PR}.0" - -LIC_FILES_CHKSUM = "file://LICENSE;md5=b6ec515b22618f55fa07276b897bacea" - -# TODO consolidate patch set -SRC_URI[md5sum] = "f5a99f765e765336a3ebbb2a24ca2be3" -SRC_URI[sha256sum] = "f55cde04f521f273c7cba08912921cc5642cfc15ca7b22d5829f0aff4371155f" - -# exclude pre-releases for both python 2.x and 3.x -UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" - -CVE_PRODUCT = "python" - -PYTHON_MAJMIN = "3.5" -PYTHON_BINABI = "${PYTHON_MAJMIN}m" - -S = "${WORKDIR}/Python-${PV}" - -inherit autotools bluetooth pkgconfig - -EXTRA_OECONF = "\ - --with-threads \ - --with-pymalloc \ - --without-cxx-main \ - --with-signal-module \ - --enable-shared \ - --enable-ipv6=${@bb.utils.contains('DISTRO_FEATURES', 'ipv6', 'yes', 'no', d)} \ -" - -PACKAGECONFIG[bluetooth] = ",ac_cv_header_bluetooth_bluetooth_h=no ac_cv_header_bluetooth_h=no,${BLUEZ}" - -do_install_append () { - sed -i -e 's:${HOSTTOOLS_DIR}/install:install:g' \ - -e 's:${HOSTTOOLS_DIR}/mkdir:mkdir:g' \ - ${D}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py -} diff --git a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch deleted file mode 100644 index 8ea3f03fe03..00000000000 --- a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch +++ /dev/null @@ -1,38 +0,0 @@ -From 04df959365e2b54d7503edf0e5534ff094284f2d Mon Sep 17 00:00:00 2001 -From: Alexander Kanavin <alex.kanavin@gmail.com> -Date: Fri, 23 Oct 2015 12:25:09 +0300 -Subject: [PATCH] Do not use the shell version of python-config that was - introduced in 3.4 - -Revert instead to the original python version: it has our tweaks and -outputs directories correctly. - -Upstream-Status: Inappropriate [oe-specific] -Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> ---- - Makefile.pre.in | 9 +++------ - 1 file changed, 3 insertions(+), 6 deletions(-) - -diff --git a/Makefile.pre.in b/Makefile.pre.in -index 236f005..5c4337f 100644 ---- a/Makefile.pre.in -+++ b/Makefile.pre.in -@@ -1348,12 +1348,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh - sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py - # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} - LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config -- # On Darwin, always use the python version of the script, the shell -- # version doesn't use the compiler customizations that are provided -- # in python (_osx_support.py). -- if test `uname -s` = Darwin; then \ -- cp python-config.py python-config; \ -- fi -+ # In OpenEmbedded, always use the python version of the script, the shell -+ # version is broken in multiple ways, and doesn't return correct directories -+ cp python-config.py python-config - - - # Install the include files --- -2.11.0 - diff --git a/meta/recipes-devtools/python/python3/0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch b/meta/recipes-devtools/python/python3/0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch deleted file mode 100644 index d1c92e9eed8..00000000000 --- a/meta/recipes-devtools/python/python3/0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch +++ /dev/null @@ -1,66 +0,0 @@ -From bcddbf40c7f1b80336268cdddacc17369fb0ccea Mon Sep 17 00:00:00 2001 -From: Libin Dang <libin.dang@windriver.com> -Date: Tue, 11 Apr 2017 14:12:15 +0800 -Subject: [PATCH] Issue #21272: Use _sysconfigdata.py to initialize - distutils.sysconfig - -Backport upstream commit -https://github.com/python/cpython/commit/409482251b06fe75c4ee56e85ffbb4b23d934159 - -Upstream-Status: Backport - -Signed-off-by: Li Zhou <li.zhou@windriver.com> ---- - Lib/distutils/sysconfig.py | 35 ++++------------------------------- - 1 file changed, 4 insertions(+), 31 deletions(-) - -diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py -index 6d5cfd0..9925d24 100644 ---- a/Lib/distutils/sysconfig.py -+++ b/Lib/distutils/sysconfig.py -@@ -424,38 +424,11 @@ _config_vars = None - - def _init_posix(): - """Initialize the module as appropriate for POSIX systems.""" -- g = {} -- # load the installed Makefile: -- try: -- filename = get_makefile_filename() -- parse_makefile(filename, g) -- except OSError as msg: -- my_msg = "invalid Python installation: unable to open %s" % filename -- if hasattr(msg, "strerror"): -- my_msg = my_msg + " (%s)" % msg.strerror -- -- raise DistutilsPlatformError(my_msg) -- -- # load the installed pyconfig.h: -- try: -- filename = get_config_h_filename() -- with open(filename) as file: -- parse_config_h(file, g) -- except OSError as msg: -- my_msg = "invalid Python installation: unable to open %s" % filename -- if hasattr(msg, "strerror"): -- my_msg = my_msg + " (%s)" % msg.strerror -- -- raise DistutilsPlatformError(my_msg) -- -- # On AIX, there are wrong paths to the linker scripts in the Makefile -- # -- these paths are relative to the Python source, but when installed -- # the scripts are in another directory. -- if python_build: -- g['LDSHARED'] = g['BLDSHARED'] -- -+ # _sysconfigdata is generated at build time, see the sysconfig module -+ from _sysconfigdata import build_time_vars - global _config_vars -- _config_vars = g -+ _config_vars = {} -+ _config_vars.update(build_time_vars) - - - def _init_nt(): --- -1.8.3.1 - diff --git a/meta/recipes-devtools/python/python3/0001-Issue-28043-SSLContext-has-improved-default-settings.patch b/meta/recipes-devtools/python/python3/0001-Issue-28043-SSLContext-has-improved-default-settings.patch deleted file mode 100644 index 321b4afa12c..00000000000 --- a/meta/recipes-devtools/python/python3/0001-Issue-28043-SSLContext-has-improved-default-settings.patch +++ /dev/null @@ -1,272 +0,0 @@ -From 758e7463c104f71b810c8588166747eeab6148d7 Mon Sep 17 00:00:00 2001 -From: Christian Heimes <christian@python.org> -Date: Sat, 10 Sep 2016 22:43:48 +0200 -Subject: [PATCH 1/4] Issue 28043: SSLContext has improved default settings - -The options OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE, OP_NO_SSLv2 (except for PROTOCOL_SSLv2), and OP_NO_SSLv3 (except for PROTOCOL_SSLv3) are set by default. The initial cipher suite list contains only HIGH ciphers, no NULL ciphers and MD5 ciphers (except for PROTOCOL_SSLv2). - -Upstream-Status: Backport -[https://github.com/python/cpython/commit/358cfd426ccc0fcd6a7940d306602138e76420ae] - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Doc/library/ssl.rst | 9 ++++++- - Lib/ssl.py | 30 +++++---------------- - Lib/test/test_ssl.py | 62 +++++++++++++++++++++++--------------------- - Modules/_ssl.c | 31 ++++++++++++++++++++++ - 4 files changed, 78 insertions(+), 54 deletions(-) - -diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst -index a2f008346b..14f2d68217 100644 ---- a/Doc/library/ssl.rst -+++ b/Doc/library/ssl.rst -@@ -1151,7 +1151,14 @@ to speed up repeated connections from the same clients. - - .. versionchanged:: 3.5.3 - -- :data:`PROTOCOL_TLS` is the default value. -+ The context is created with secure default values. The options -+ :data:`OP_NO_COMPRESSION`, :data:`OP_CIPHER_SERVER_PREFERENCE`, -+ :data:`OP_SINGLE_DH_USE`, :data:`OP_SINGLE_ECDH_USE`, -+ :data:`OP_NO_SSLv2` (except for :data:`PROTOCOL_SSLv2`), -+ and :data:`OP_NO_SSLv3` (except for :data:`PROTOCOL_SSLv3`) are -+ set by default. The initial cipher suite list contains only ``HIGH`` -+ ciphers, no ``NULL`` ciphers and no ``MD5`` ciphers (except for -+ :data:`PROTOCOL_SSLv2`). - - - :class:`SSLContext` objects have the following methods and attributes: -diff --git a/Lib/ssl.py b/Lib/ssl.py -index e1913904f3..4d302a78fa 100644 ---- a/Lib/ssl.py -+++ b/Lib/ssl.py -@@ -446,32 +446,16 @@ def create_default_context(purpose=Purpose.SERVER_AUTH, *, cafile=None, - if not isinstance(purpose, _ASN1Object): - raise TypeError(purpose) - -+ # SSLContext sets OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_COMPRESSION, -+ # OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE and OP_SINGLE_ECDH_USE -+ # by default. - context = SSLContext(PROTOCOL_TLS) - -- # SSLv2 considered harmful. -- context.options |= OP_NO_SSLv2 -- -- # SSLv3 has problematic security and is only required for really old -- # clients such as IE6 on Windows XP -- context.options |= OP_NO_SSLv3 -- -- # disable compression to prevent CRIME attacks (OpenSSL 1.0+) -- context.options |= getattr(_ssl, "OP_NO_COMPRESSION", 0) -- - if purpose == Purpose.SERVER_AUTH: - # verify certs and host name in client mode - context.verify_mode = CERT_REQUIRED - context.check_hostname = True - elif purpose == Purpose.CLIENT_AUTH: -- # Prefer the server's ciphers by default so that we get stronger -- # encryption -- context.options |= getattr(_ssl, "OP_CIPHER_SERVER_PREFERENCE", 0) -- -- # Use single use keys in order to improve forward secrecy -- context.options |= getattr(_ssl, "OP_SINGLE_DH_USE", 0) -- context.options |= getattr(_ssl, "OP_SINGLE_ECDH_USE", 0) -- -- # disallow ciphers with known vulnerabilities - context.set_ciphers(_RESTRICTED_SERVER_CIPHERS) - - if cafile or capath or cadata: -@@ -497,12 +481,10 @@ def _create_unverified_context(protocol=PROTOCOL_TLS, *, cert_reqs=None, - if not isinstance(purpose, _ASN1Object): - raise TypeError(purpose) - -+ # SSLContext sets OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_COMPRESSION, -+ # OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE and OP_SINGLE_ECDH_USE -+ # by default. - context = SSLContext(protocol) -- # SSLv2 considered harmful. -- context.options |= OP_NO_SSLv2 -- # SSLv3 has problematic security and is only required for really old -- # clients such as IE6 on Windows XP -- context.options |= OP_NO_SSLv3 - - if cert_reqs is not None: - context.verify_mode = cert_reqs -diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py -index ffb7314f57..f91af7bd05 100644 ---- a/Lib/test/test_ssl.py -+++ b/Lib/test/test_ssl.py -@@ -73,6 +73,12 @@ NULLBYTECERT = data_file("nullbytecert.pem") - DHFILE = data_file("dh1024.pem") - BYTES_DHFILE = os.fsencode(DHFILE) - -+# Not defined in all versions of OpenSSL -+OP_NO_COMPRESSION = getattr(ssl, "OP_NO_COMPRESSION", 0) -+OP_SINGLE_DH_USE = getattr(ssl, "OP_SINGLE_DH_USE", 0) -+OP_SINGLE_ECDH_USE = getattr(ssl, "OP_SINGLE_ECDH_USE", 0) -+OP_CIPHER_SERVER_PREFERENCE = getattr(ssl, "OP_CIPHER_SERVER_PREFERENCE", 0) -+ - - def handle_error(prefix): - exc_format = ' '.join(traceback.format_exception(*sys.exc_info())) -@@ -839,8 +845,9 @@ class ContextTests(unittest.TestCase): - ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) - # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value - default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) -- if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0): -- default |= ssl.OP_NO_COMPRESSION -+ # SSLContext also enables these by default -+ default |= (OP_NO_COMPRESSION | OP_CIPHER_SERVER_PREFERENCE | -+ OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE) - self.assertEqual(default, ctx.options) - ctx.options |= ssl.OP_NO_TLSv1 - self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) -@@ -1205,16 +1212,29 @@ class ContextTests(unittest.TestCase): - stats["x509"] += 1 - self.assertEqual(ctx.cert_store_stats(), stats) - -+ def _assert_context_options(self, ctx): -+ self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -+ if OP_NO_COMPRESSION != 0: -+ self.assertEqual(ctx.options & OP_NO_COMPRESSION, -+ OP_NO_COMPRESSION) -+ if OP_SINGLE_DH_USE != 0: -+ self.assertEqual(ctx.options & OP_SINGLE_DH_USE, -+ OP_SINGLE_DH_USE) -+ if OP_SINGLE_ECDH_USE != 0: -+ self.assertEqual(ctx.options & OP_SINGLE_ECDH_USE, -+ OP_SINGLE_ECDH_USE) -+ if OP_CIPHER_SERVER_PREFERENCE != 0: -+ self.assertEqual(ctx.options & OP_CIPHER_SERVER_PREFERENCE, -+ OP_CIPHER_SERVER_PREFERENCE) -+ - def test_create_default_context(self): - ctx = ssl.create_default_context() -+ - self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) - self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) - self.assertTrue(ctx.check_hostname) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -- self.assertEqual( -- ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), -- getattr(ssl, "OP_NO_COMPRESSION", 0), -- ) -+ self._assert_context_options(ctx) -+ - - with open(SIGNING_CA) as f: - cadata = f.read() -@@ -1222,40 +1242,24 @@ class ContextTests(unittest.TestCase): - cadata=cadata) - self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) - self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -- self.assertEqual( -- ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), -- getattr(ssl, "OP_NO_COMPRESSION", 0), -- ) -+ self._assert_context_options(ctx) - - ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) - self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) - self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -- self.assertEqual( -- ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), -- getattr(ssl, "OP_NO_COMPRESSION", 0), -- ) -- self.assertEqual( -- ctx.options & getattr(ssl, "OP_SINGLE_DH_USE", 0), -- getattr(ssl, "OP_SINGLE_DH_USE", 0), -- ) -- self.assertEqual( -- ctx.options & getattr(ssl, "OP_SINGLE_ECDH_USE", 0), -- getattr(ssl, "OP_SINGLE_ECDH_USE", 0), -- ) -+ self._assert_context_options(ctx) - - def test__create_stdlib_context(self): - ctx = ssl._create_stdlib_context() - self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) - self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) - self.assertFalse(ctx.check_hostname) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -+ self._assert_context_options(ctx) - - ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1) - self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1) - self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -+ self._assert_context_options(ctx) - - ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1, - cert_reqs=ssl.CERT_REQUIRED, -@@ -1263,12 +1267,12 @@ class ContextTests(unittest.TestCase): - self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1) - self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) - self.assertTrue(ctx.check_hostname) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -+ self._assert_context_options(ctx) - - ctx = ssl._create_stdlib_context(purpose=ssl.Purpose.CLIENT_AUTH) - self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) - self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) -- self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) -+ self._assert_context_options(ctx) - - def test_check_hostname(self): - ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) -diff --git a/Modules/_ssl.c b/Modules/_ssl.c -index 86482677ae..0d5c121d2c 100644 ---- a/Modules/_ssl.c -+++ b/Modules/_ssl.c -@@ -2330,6 +2330,7 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) - PySSLContext *self; - long options; - SSL_CTX *ctx = NULL; -+ int result; - #if defined(SSL_MODE_RELEASE_BUFFERS) - unsigned long libver; - #endif -@@ -2393,8 +2394,38 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) - options |= SSL_OP_NO_SSLv2; - if (proto_version != PY_SSL_VERSION_SSL3) - options |= SSL_OP_NO_SSLv3; -+ /* Minimal security flags for server and client side context. -+ * Client sockets ignore server-side parameters. */ -+#ifdef SSL_OP_NO_COMPRESSION -+ options |= SSL_OP_NO_COMPRESSION; -+#endif -+#ifdef SSL_OP_CIPHER_SERVER_PREFERENCE -+ options |= SSL_OP_CIPHER_SERVER_PREFERENCE; -+#endif -+#ifdef SSL_OP_SINGLE_DH_USE -+ options |= SSL_OP_SINGLE_DH_USE; -+#endif -+#ifdef SSL_OP_SINGLE_ECDH_USE -+ options |= SSL_OP_SINGLE_ECDH_USE; -+#endif - SSL_CTX_set_options(self->ctx, options); - -+ /* A bare minimum cipher list without completly broken cipher suites. -+ * It's far from perfect but gives users a better head start. */ -+ if (proto_version != PY_SSL_VERSION_SSL2) { -+ result = SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL:!MD5"); -+ } else { -+ /* SSLv2 needs MD5 */ -+ result = SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL"); -+ } -+ if (result == 0) { -+ Py_DECREF(self); -+ ERR_clear_error(); -+ PyErr_SetString(PySSLErrorObject, -+ "No cipher can be selected."); -+ return NULL; -+ } -+ - #if defined(SSL_MODE_RELEASE_BUFFERS) - /* Set SSL_MODE_RELEASE_BUFFERS. This potentially greatly reduces memory - usage for no cost at all. However, don't do this for OpenSSL versions --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch b/meta/recipes-devtools/python/python3/0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch deleted file mode 100644 index dd6b24fe922..00000000000 --- a/meta/recipes-devtools/python/python3/0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch +++ /dev/null @@ -1,37 +0,0 @@ -From 2f5a4c708d90fa8db21f446ae879cff79387448d Mon Sep 17 00:00:00 2001 -From: Khem Raj <raj.khem@gmail.com> -Date: Mon, 20 May 2013 21:03:16 -0700 -Subject: [PATCH] h2py: Fix issue 13032 where it fails with UnicodeDecodeError - -use utf-8 to open the files - -Signed-off-by: Khem Raj <raj.khem@gmail.com> - -Upstream-Status: Pending ---- - Tools/scripts/h2py.py | 4 ++-- - 1 file changed, 2 insertions(+), 2 deletions(-) - -diff --git a/Tools/scripts/h2py.py b/Tools/scripts/h2py.py -index 4f871d9..a53fbe0 100755 ---- a/Tools/scripts/h2py.py -+++ b/Tools/scripts/h2py.py -@@ -69,13 +69,13 @@ def main(): - sys.stdout.write('# Generated by h2py from stdin\n') - process(sys.stdin, sys.stdout) - else: -- fp = open(filename, 'r') -+ fp = open(filename, 'r', encoding='utf-8') - outfile = os.path.basename(filename) - i = outfile.rfind('.') - if i > 0: outfile = outfile[:i] - modname = outfile.upper() - outfile = modname + '.py' -- outfp = open(outfile, 'w') -+ outfp = open(outfile, 'w', encoding='utf-8') - outfp.write('# Generated by h2py from %s\n' % filename) - filedict = {} - for dir in searchdirs: --- -1.8.1.2 - diff --git a/meta/recipes-devtools/python/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch b/meta/recipes-devtools/python/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch deleted file mode 100644 index ae473661ec7..00000000000 --- a/meta/recipes-devtools/python/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch +++ /dev/null @@ -1,140 +0,0 @@ -From 564a5cc634028970dc2f9d8ecc0e464a4fb1dcb6 Mon Sep 17 00:00:00 2001 -From: Changqing Li <changqing.li@windriver.com> -Date: Mon, 22 Oct 2018 15:19:51 +0800 -Subject: [PATCH] python3: use cc_basename to replace CC for checking compiler - -When working path contains "clang"/"gcc"/"icc", it might be part of $CC -because of the "--sysroot" parameter. That could cause judgement error -about clang/gcc/icc compilers. e.g. -When "icc" is containded in working path, below errors are reported when -compiling python3: -x86_64-wrs-linux-gcc: error: strict: No such file or directory -x86_64-wrs-linux-gcc: error: unrecognized command line option '-fp-model' - -Here use cc_basename to replace CC for checking compiler to avoid such -kind of issue. - -Upstream-Status: Pending - -Signed-off-by: Li Zhou <li.zhou@windriver.com> - -patch originally from Li Zhou, I just rework it to new version - -Signed-off-by: Changqing Li <changqing.li@windriver.com> ---- - configure.ac | 23 ++++++++++++----------- - 1 file changed, 12 insertions(+), 11 deletions(-) - -diff --git a/configure.ac b/configure.ac -index 95c98d1..1b9589e 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -54,6 +54,7 @@ AC_CONFIG_HEADER(pyconfig.h) - AC_CANONICAL_HOST - AC_SUBST(build) - AC_SUBST(host) -+LT_INIT - - # pybuilddir.txt will be created by --generate-posix-vars in the Makefile - rm -f pybuilddir.txt -@@ -716,7 +717,7 @@ AC_MSG_RESULT($with_cxx_main) - preset_cxx="$CXX" - if test -z "$CXX" - then -- case "$CC" in -+ case "$cc_basename" in - gcc) AC_PATH_TOOL(CXX, [g++], [g++], [notfound]) ;; - cc) AC_PATH_TOOL(CXX, [c++], [c++], [notfound]) ;; - clang|*/clang) AC_PATH_TOOL(CXX, [clang++], [clang++], [notfound]) ;; -@@ -978,14 +979,14 @@ rmdir CaseSensitiveTestDir - - case $MACHDEP in - bsdos*) -- case $CC in -+ case $cc_basename in - gcc) CC="$CC -D_HAVE_BSDI";; - esac;; - esac - - case $ac_sys_system in - hp*|HP*) -- case $CC in -+ case $cc_basename in - cc|*/cc) CC="$CC -Ae";; - esac;; - esac -@@ -1310,7 +1311,7 @@ else - fi], - [AC_MSG_RESULT(no)]) - if test "$Py_LTO" = 'true' ; then -- case $CC in -+ case $cc_basename in - *clang*) - # Any changes made here should be reflected in the GCC+Darwin case below - LTOFLAGS="-flto" -@@ -1374,7 +1375,7 @@ then - fi - fi - LLVM_PROF_ERR=no --case $CC in -+case $cc_basename in - *clang*) - # Any changes made here should be reflected in the GCC+Darwin case below - PGO_PROF_GEN_FLAG="-fprofile-instr-generate" -@@ -1451,7 +1452,7 @@ then - WRAP="-fwrapv" - fi - -- case $CC in -+ case $cc_basename in - *clang*) - cc_is_clang=1 - ;; -@@ -1553,7 +1554,7 @@ yes) - - # ICC doesn't recognize the option, but only emits a warning - ## XXX does it emit an unused result warning and can it be disabled? -- case "$CC" in -+ case "$cc_basename" in - *icc*) - ac_cv_disable_unused_result_warning=no - ;; -@@ -1808,7 +1809,7 @@ yes) - esac - - # ICC needs -fp-model strict or floats behave badly --case "$CC" in -+case "$cc_basename" in - *icc*) - CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" - ;; -@@ -2574,7 +2575,7 @@ then - then CCSHARED="-fPIC" - else CCSHARED="-Kpic -belf" - fi;; -- IRIX*/6*) case $CC in -+ IRIX*/6*) case $cc_basename in - *gcc*) CCSHARED="-shared";; - *) CCSHARED="";; - esac;; -@@ -2615,7 +2616,7 @@ then - then - LINKFORSHARED="-Wl,--export-dynamic" - fi;; -- SunOS/5*) case $CC in -+ SunOS/5*) case $cc_basename in - *gcc*) - if $CC -Xlinker --help 2>&1 | grep export-dynamic >/dev/null - then -@@ -5187,7 +5188,7 @@ if test "$have_gcc_asm_for_x87" = yes; then - # Some versions of gcc miscompile inline asm: - # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46491 - # http://gcc.gnu.org/ml/gcc/2010-11/msg00366.html -- case $CC in -+ case $cc_basename in - *gcc*) - AC_MSG_CHECKING(for gcc ipa-pure-const bug) - saved_cflags="$CFLAGS" --- -2.7.4 - diff --git a/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch b/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch deleted file mode 100644 index 2b4ba316e45..00000000000 --- a/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch +++ /dev/null @@ -1,40 +0,0 @@ -From 98586d6dc598e40b8b821b0dde57599e188a7ca4 Mon Sep 17 00:00:00 2001 -From: Anuj Mittal <anuj.mittal@intel.com> -Date: Tue, 7 Aug 2018 16:43:17 +0800 -Subject: [PATCH 2/2] Makefile: add target to split profile generation - -We don't want to have profile task invoked from here and want to use -qemu-user instead. Split the profile-opt task so qemu can be invoked -once binaries have been built with instrumentation and then we can go -ahead and build again using the profile data generated. - -Upstream-Status: Inappropriate [OE-specific] - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Makefile.pre.in | 6 ++---- - 1 file changed, 2 insertions(+), 4 deletions(-) - -diff --git a/Makefile.pre.in b/Makefile.pre.in -index 84bc3ff..017a2c4 100644 ---- a/Makefile.pre.in -+++ b/Makefile.pre.in -@@ -469,13 +469,12 @@ profile-opt: - $(MAKE) profile-removal - $(MAKE) build_all_generate_profile - $(MAKE) profile-removal -- @echo "Running code to generate profile data (this can take a while):" -- $(MAKE) run_profile_task -- $(MAKE) build_all_merge_profile -+ -+clean_and_use_profile: - @echo "Rebuilding with profile guided optimizations:" - $(MAKE) clean - $(MAKE) build_all_use_profile - $(MAKE) profile-removal - - build_all_generate_profile: - $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch b/meta/recipes-devtools/python/python3/0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch deleted file mode 100644 index d48cad7586b..00000000000 --- a/meta/recipes-devtools/python/python3/0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch +++ /dev/null @@ -1,227 +0,0 @@ -From e950ea68dab006944af194c9910b8f2341d1437d Mon Sep 17 00:00:00 2001 -From: Christian Heimes <christian@python.org> -Date: Thu, 7 Sep 2017 20:23:52 -0700 -Subject: [PATCH] bpo-29136: Add TLS 1.3 cipher suites and OP_NO_TLSv1_3 - (GH-1363) (#3444) - -* bpo-29136: Add TLS 1.3 support - -TLS 1.3 introduces a new, distinct set of cipher suites. The TLS 1.3 -cipher suites don't overlap with cipher suites from TLS 1.2 and earlier. -Since Python sets its own set of permitted ciphers, TLS 1.3 handshake -will fail as soon as OpenSSL 1.1.1 is released. Let's enable the common -AES-GCM and ChaCha20 suites. - -Additionally the flag OP_NO_TLSv1_3 is added. It defaults to 0 (no op) with -OpenSSL prior to 1.1.1. This allows applications to opt-out from TLS 1.3 -now. - -Signed-off-by: Christian Heimes <christian@python.org>. -(cherry picked from commit cb5b68abdeb1b1d56c581d5b4d647018703d61e3) - -Upstream-Status: Backport -[https://github.com/python/cpython/commit/cb5b68abdeb1b1d56c581d5b4d647018703d61e3] - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Doc/library/ssl.rst | 21 ++++++++++++++ - Lib/ssl.py | 7 +++++ - Lib/test/test_ssl.py | 29 ++++++++++++++++++- - .../2017-09-04-16-39-49.bpo-29136.vSn1oR.rst | 1 + - Modules/_ssl.c | 13 +++++++++ - 5 files changed, 70 insertions(+), 1 deletion(-) - create mode 100644 Misc/NEWS.d/next/Library/2017-09-04-16-39-49.bpo-29136.vSn1oR.rst - -diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst -index 14f2d68217..29c5e94cf6 100644 ---- a/Doc/library/ssl.rst -+++ b/Doc/library/ssl.rst -@@ -285,6 +285,11 @@ purposes. - - 3DES was dropped from the default cipher string. - -+ .. versionchanged:: 3.7 -+ -+ TLS 1.3 cipher suites TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, -+ and TLS_CHACHA20_POLY1305_SHA256 were added to the default cipher string. -+ - - Random generation - ^^^^^^^^^^^^^^^^^ -@@ -719,6 +724,16 @@ Constants - - .. versionadded:: 3.4 - -+.. data:: OP_NO_TLSv1_3 -+ -+ Prevents a TLSv1.3 connection. This option is only applicable in conjunction -+ with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.3 as -+ the protocol version. TLS 1.3 is available with OpenSSL 1.1.1 or later. -+ When Python has been compiled against an older version of OpenSSL, the -+ flag defaults to *0*. -+ -+ .. versionadded:: 3.7 -+ - .. data:: OP_CIPHER_SERVER_PREFERENCE - - Use the server's cipher ordering preference, rather than the client's. -@@ -783,6 +798,12 @@ Constants - - .. versionadded:: 3.3 - -+.. data:: HAS_TLSv1_3 -+ -+ Whether the OpenSSL library has built-in support for the TLS 1.3 protocol. -+ -+ .. versionadded:: 3.7 -+ - .. data:: CHANNEL_BINDING_TYPES - - List of supported TLS channel binding types. Strings in this list -diff --git a/Lib/ssl.py b/Lib/ssl.py -index 4d302a78fa..f233e72e1f 100644 ---- a/Lib/ssl.py -+++ b/Lib/ssl.py -@@ -122,6 +122,7 @@ _import_symbols('OP_') - _import_symbols('ALERT_DESCRIPTION_') - _import_symbols('SSL_ERROR_') - _import_symbols('VERIFY_') -+from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN, HAS_TLSv1_3 - - from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN - -@@ -162,6 +163,7 @@ else: - # (OpenSSL's default setting is 'DEFAULT:!aNULL:!eNULL') - # Enable a better set of ciphers by default - # This list has been explicitly chosen to: -+# * TLS 1.3 ChaCha20 and AES-GCM cipher suites - # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) - # * Prefer ECDHE over DHE for better performance - # * Prefer AEAD over CBC for better performance and security -@@ -173,6 +175,8 @@ else: - # * Disable NULL authentication, NULL encryption, 3DES and MD5 MACs - # for security reasons - _DEFAULT_CIPHERS = ( -+ 'TLS13-AES-256-GCM-SHA384:TLS13-CHACHA20-POLY1305-SHA256:' -+ 'TLS13-AES-128-GCM-SHA256:' - 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' - 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' - '!aNULL:!eNULL:!MD5:!3DES' -@@ -180,6 +184,7 @@ _DEFAULT_CIPHERS = ( - - # Restricted and more secure ciphers for the server side - # This list has been explicitly chosen to: -+# * TLS 1.3 ChaCha20 and AES-GCM cipher suites - # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) - # * Prefer ECDHE over DHE for better performance - # * Prefer AEAD over CBC for better performance and security -@@ -190,6 +195,8 @@ _DEFAULT_CIPHERS = ( - # * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, RC4, and - # 3DES for security reasons - _RESTRICTED_SERVER_CIPHERS = ( -+ 'TLS13-AES-256-GCM-SHA384:TLS13-CHACHA20-POLY1305-SHA256:' -+ 'TLS13-AES-128-GCM-SHA256:' - 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' - 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' - '!aNULL:!eNULL:!MD5:!DSS:!RC4:!3DES' -diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py -index f91af7bd05..1acc12ec2d 100644 ---- a/Lib/test/test_ssl.py -+++ b/Lib/test/test_ssl.py -@@ -150,6 +150,13 @@ class BasicSocketTests(unittest.TestCase): - ssl.OP_NO_COMPRESSION - self.assertIn(ssl.HAS_SNI, {True, False}) - self.assertIn(ssl.HAS_ECDH, {True, False}) -+ ssl.OP_NO_SSLv2 -+ ssl.OP_NO_SSLv3 -+ ssl.OP_NO_TLSv1 -+ ssl.OP_NO_TLSv1_3 -+ if ssl.OPENSSL_VERSION_INFO >= (1, 0, 1): -+ ssl.OP_NO_TLSv1_1 -+ ssl.OP_NO_TLSv1_2 - - def test_str_for_enums(self): - # Make sure that the PROTOCOL_* constants have enum-like string -@@ -3028,12 +3035,33 @@ else: - self.assertEqual(s.version(), 'TLSv1') - self.assertIs(s.version(), None) - -+ @unittest.skipUnless(ssl.HAS_TLSv1_3, -+ "test requires TLSv1.3 enabled OpenSSL") -+ def test_tls1_3(self): -+ context = ssl.SSLContext(ssl.PROTOCOL_TLS) -+ context.load_cert_chain(CERTFILE) -+ # disable all but TLS 1.3 -+ context.options |= ( -+ ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1_2 -+ ) -+ with ThreadedEchoServer(context=context) as server: -+ with context.wrap_socket(socket.socket()) as s: -+ s.connect((HOST, server.port)) -+ self.assertIn(s.cipher()[0], [ -+ 'TLS13-AES-256-GCM-SHA384', -+ 'TLS13-CHACHA20-POLY1305-SHA256', -+ 'TLS13-AES-128-GCM-SHA256', -+ ]) -+ - @unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL") - def test_default_ecdh_curve(self): - # Issue #21015: elliptic curve-based Diffie Hellman key exchange - # should be enabled by default on SSL contexts. - context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) - context.load_cert_chain(CERTFILE) -+ # TLSv1.3 defaults to PFS key agreement and no longer has KEA in -+ # cipher name. -+ context.options |= ssl.OP_NO_TLSv1_3 - # Prior to OpenSSL 1.0.0, ECDH ciphers have to be enabled - # explicitly using the 'ECCdraft' cipher alias. Otherwise, - # our default cipher list should prefer ECDH-based ciphers -@@ -3394,7 +3422,6 @@ else: - s.sendfile(file) - self.assertEqual(s.recv(1024), TEST_DATA) - -- - def test_main(verbose=False): - if support.verbose: - import warnings -diff --git a/Misc/NEWS.d/next/Library/2017-09-04-16-39-49.bpo-29136.vSn1oR.rst b/Misc/NEWS.d/next/Library/2017-09-04-16-39-49.bpo-29136.vSn1oR.rst -new file mode 100644 -index 0000000000..e76997ef83 ---- /dev/null -+++ b/Misc/NEWS.d/next/Library/2017-09-04-16-39-49.bpo-29136.vSn1oR.rst -@@ -0,0 +1 @@ -+Add TLS 1.3 cipher suites and OP_NO_TLSv1_3. -diff --git a/Modules/_ssl.c b/Modules/_ssl.c -index 0d5c121d2c..c71d89607c 100644 ---- a/Modules/_ssl.c -+++ b/Modules/_ssl.c -@@ -4842,6 +4842,11 @@ PyInit__ssl(void) - #if HAVE_TLSv1_2 - PyModule_AddIntConstant(m, "OP_NO_TLSv1_1", SSL_OP_NO_TLSv1_1); - PyModule_AddIntConstant(m, "OP_NO_TLSv1_2", SSL_OP_NO_TLSv1_2); -+#endif -+#ifdef SSL_OP_NO_TLSv1_3 -+ PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", SSL_OP_NO_TLSv1_3); -+#else -+ PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", 0); - #endif - PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE", - SSL_OP_CIPHER_SERVER_PREFERENCE); -@@ -4890,6 +4895,14 @@ PyInit__ssl(void) - Py_INCREF(r); - PyModule_AddObject(m, "HAS_ALPN", r); - -+#if defined(TLS1_3_VERSION) && !defined(OPENSSL_NO_TLS1_3) -+ r = Py_True; -+#else -+ r = Py_False; -+#endif -+ Py_INCREF(r); -+ PyModule_AddObject(m, "HAS_TLSv1_3", r); -+ - /* Mappings for error codes */ - err_codes_to_names = PyDict_New(); - err_names_to_codes = PyDict_New(); --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch b/meta/recipes-devtools/python/python3/0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch deleted file mode 100644 index 56d591d1b52..00000000000 --- a/meta/recipes-devtools/python/python3/0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch +++ /dev/null @@ -1,173 +0,0 @@ -From 170a614904febd14ff6cfd7a75c9bccc114b3948 Mon Sep 17 00:00:00 2001 -From: Christian Heimes <christian@python.org> -Date: Tue, 14 Aug 2018 16:56:32 +0200 -Subject: [PATCH] bpo-32947: Fixes for TLS 1.3 and OpenSSL 1.1.1 (GH-8761) - -Backport of TLS 1.3 related fixes from 3.7. - -Misc fixes and workarounds for compatibility with OpenSSL 1.1.1 from git -master and TLS 1.3 support. With OpenSSL 1.1.1, Python negotiates TLS 1.3 by -default. Some test cases only apply to TLS 1.2. - -OpenSSL 1.1.1 has added a new option OP_ENABLE_MIDDLEBOX_COMPAT for TLS -1.3. The feature is enabled by default for maximum compatibility with -broken middle boxes. Users should be able to disable the hack and CPython's test suite needs -it to verify default options - -Signed-off-by: Christian Heimes <christian@python.org> - -Upstream-Status: Backport -[https://github.com/python/cpython/commit/2a4ee8aa01d61b6a9c8e9c65c211e61bdb471826] - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Doc/library/ssl.rst | 9 ++++++ - Lib/test/test_asyncio/test_events.py | 6 +++- - Lib/test/test_ssl.py | 29 +++++++++++++++---- - .../2018-08-14-08-57-01.bpo-32947.mqStVW.rst | 2 ++ - Modules/_ssl.c | 4 +++ - 5 files changed, 44 insertions(+), 6 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2018-08-14-08-57-01.bpo-32947.mqStVW.rst - -diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst -index 29c5e94cf6..f63a3deec5 100644 ---- a/Doc/library/ssl.rst -+++ b/Doc/library/ssl.rst -@@ -757,6 +757,15 @@ Constants - - .. versionadded:: 3.3 - -+.. data:: OP_ENABLE_MIDDLEBOX_COMPAT -+ -+ Send dummy Change Cipher Spec (CCS) messages in TLS 1.3 handshake to make -+ a TLS 1.3 connection look more like a TLS 1.2 connection. -+ -+ This option is only available with OpenSSL 1.1.1 and later. -+ -+ .. versionadded:: 3.6.7 -+ - .. data:: OP_NO_COMPRESSION - - Disable compression on the SSL channel. This is useful if the application -diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py -index 492a84a231..6f208474b9 100644 ---- a/Lib/test/test_asyncio/test_events.py -+++ b/Lib/test/test_asyncio/test_events.py -@@ -1169,7 +1169,11 @@ class EventLoopTestsMixin: - self.loop.run_until_complete(f_c) - - # close connection -- proto.transport.close() -+ # transport may be None with TLS 1.3, because connection is -+ # interrupted, server is unable to send session tickets, and -+ # transport is closed. -+ if proto.transport is not None: -+ proto.transport.close() - server.close() - - def test_legacy_create_server_ssl_match_failed(self): -diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py -index 1acc12ec2d..a2e1d32a62 100644 ---- a/Lib/test/test_ssl.py -+++ b/Lib/test/test_ssl.py -@@ -78,6 +78,7 @@ OP_NO_COMPRESSION = getattr(ssl, "OP_NO_COMPRESSION", 0) - OP_SINGLE_DH_USE = getattr(ssl, "OP_SINGLE_DH_USE", 0) - OP_SINGLE_ECDH_USE = getattr(ssl, "OP_SINGLE_ECDH_USE", 0) - OP_CIPHER_SERVER_PREFERENCE = getattr(ssl, "OP_CIPHER_SERVER_PREFERENCE", 0) -+OP_ENABLE_MIDDLEBOX_COMPAT = getattr(ssl, "OP_ENABLE_MIDDLEBOX_COMPAT", 0) - - - def handle_error(prefix): -@@ -155,8 +156,8 @@ class BasicSocketTests(unittest.TestCase): - ssl.OP_NO_TLSv1 - ssl.OP_NO_TLSv1_3 - if ssl.OPENSSL_VERSION_INFO >= (1, 0, 1): -- ssl.OP_NO_TLSv1_1 -- ssl.OP_NO_TLSv1_2 -+ ssl.OP_NO_TLSv1_1 -+ ssl.OP_NO_TLSv1_2 - - def test_str_for_enums(self): - # Make sure that the PROTOCOL_* constants have enum-like string -@@ -854,7 +855,8 @@ class ContextTests(unittest.TestCase): - default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) - # SSLContext also enables these by default - default |= (OP_NO_COMPRESSION | OP_CIPHER_SERVER_PREFERENCE | -- OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE) -+ OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE | -+ OP_ENABLE_MIDDLEBOX_COMPAT) - self.assertEqual(default, ctx.options) - ctx.options |= ssl.OP_NO_TLSv1 - self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) -@@ -1860,11 +1862,26 @@ else: - self.sock, server_side=True) - self.server.selected_npn_protocols.append(self.sslconn.selected_npn_protocol()) - self.server.selected_alpn_protocols.append(self.sslconn.selected_alpn_protocol()) -- except (ssl.SSLError, ConnectionResetError) as e: -+ except (ConnectionResetError, BrokenPipeError) as e: - # We treat ConnectionResetError as though it were an - # SSLError - OpenSSL on Ubuntu abruptly closes the - # connection when asked to use an unsupported protocol. - # -+ # BrokenPipeError is raised in TLS 1.3 mode, when OpenSSL -+ # tries to send session tickets after handshake. -+ # https://github.com/openssl/openssl/issues/6342 -+ self.server.conn_errors.append(str(e)) -+ if self.server.chatty: -+ handle_error( -+ "\n server: bad connection attempt from " + repr( -+ self.addr) + ":\n") -+ self.running = False -+ self.close() -+ return False -+ except (ssl.SSLError, OSError) as e: -+ # OSError may occur with wrong protocols, e.g. both -+ # sides use PROTOCOL_TLS_SERVER. -+ # - # XXX Various errors can have happened here, for example - # a mismatching protocol version, an invalid certificate, - # or a low-level bug. This should be made more discriminating. -@@ -2974,7 +2991,7 @@ else: - # Block on the accept and wait on the connection to close. - evt.set() - remote, peer = server.accept() -- remote.recv(1) -+ remote.send(remote.recv(4)) - - t = threading.Thread(target=serve) - t.start() -@@ -2982,6 +2999,8 @@ else: - evt.wait() - client = context.wrap_socket(socket.socket()) - client.connect((host, port)) -+ client.send(b'data') -+ client.recv() - client_addr = client.getsockname() - client.close() - t.join() -diff --git a/Misc/NEWS.d/next/Library/2018-08-14-08-57-01.bpo-32947.mqStVW.rst b/Misc/NEWS.d/next/Library/2018-08-14-08-57-01.bpo-32947.mqStVW.rst -new file mode 100644 -index 0000000000..28de360c36 ---- /dev/null -+++ b/Misc/NEWS.d/next/Library/2018-08-14-08-57-01.bpo-32947.mqStVW.rst -@@ -0,0 +1,2 @@ -+Add OP_ENABLE_MIDDLEBOX_COMPAT and test workaround for TLSv1.3 for future -+compatibility with OpenSSL 1.1.1. -diff --git a/Modules/_ssl.c b/Modules/_ssl.c -index c71d89607c..eb123a87ba 100644 ---- a/Modules/_ssl.c -+++ b/Modules/_ssl.c -@@ -4858,6 +4858,10 @@ PyInit__ssl(void) - PyModule_AddIntConstant(m, "OP_NO_COMPRESSION", - SSL_OP_NO_COMPRESSION); - #endif -+#ifdef SSL_OP_ENABLE_MIDDLEBOX_COMPAT -+ PyModule_AddIntConstant(m, "OP_ENABLE_MIDDLEBOX_COMPAT", -+ SSL_OP_ENABLE_MIDDLEBOX_COMPAT); -+#endif - - #if HAVE_SNI - r = Py_True; --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch b/meta/recipes-devtools/python/python3/0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch deleted file mode 100644 index b97d5501e18..00000000000 --- a/meta/recipes-devtools/python/python3/0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch +++ /dev/null @@ -1,110 +0,0 @@ -From 0c9354362bfa5f90fbea8ff8237a1f1f5dba686f Mon Sep 17 00:00:00 2001 -From: Christian Heimes <christian@python.org> -Date: Wed, 12 Sep 2018 15:20:31 +0800 -Subject: [PATCH] bpo-33570: TLS 1.3 ciphers for OpenSSL 1.1.1 (GH-6976) - -Change TLS 1.3 cipher suite settings for compatibility with OpenSSL -1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by -default. - -Also update multissltests and Travis config to test with latest OpenSSL. - -Signed-off-by: Christian Heimes <christian@python.org> -(cherry picked from commit e8eb6cb7920ded66abc5d284319a8539bdc2bae3) - -Co-authored-by: Christian Heimes <christian@python.org - -Upstream-Status: Backport -[https://github.com/python/cpython/commit/3e630c541b35c96bfe5619165255e559f577ee71] - -Tweaked patch to not take changes for multissltests and Travis config. - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Lib/test/test_ssl.py | 51 ++++++++++++++++++++++---------------------- - 1 file changed, 26 insertions(+), 25 deletions(-) - -diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py -index a2e1d32a62..c484ead5ff 100644 ---- a/Lib/test/test_ssl.py -+++ b/Lib/test/test_ssl.py -@@ -3024,17 +3024,21 @@ else: - sock.do_handshake() - self.assertEqual(cm.exception.errno, errno.ENOTCONN) - -- def test_default_ciphers(self): -- context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) -- try: -- # Force a set of weak ciphers on our client context -- context.set_ciphers("DES") -- except ssl.SSLError: -- self.skipTest("no DES cipher available") -- with ThreadedEchoServer(CERTFILE, -- ssl_version=ssl.PROTOCOL_SSLv23, -- chatty=False) as server: -- with context.wrap_socket(socket.socket()) as s: -+ def test_no_shared_ciphers(self): -+ server_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) -+ server_context.load_cert_chain(SIGNED_CERTFILE) -+ client_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) -+ client_context.verify_mode = ssl.CERT_REQUIRED -+ client_context.check_hostname = True -+ -+ client_context.set_ciphers("AES128") -+ server_context.set_ciphers("AES256") -+ # OpenSSL enables all TLS 1.3 ciphers, enforce TLS 1.2 for test -+ client_context.options |= ssl.OP_NO_TLSv1_3 -+ with ThreadedEchoServer(context=server_context) as server: -+ with client_context.wrap_socket( -+ socket.socket(), -+ server_hostname="localhost") as s: - with self.assertRaises(OSError): - s.connect((HOST, server.port)) - self.assertIn("no shared cipher", str(server.conn_errors[0])) -@@ -3067,9 +3071,9 @@ else: - with context.wrap_socket(socket.socket()) as s: - s.connect((HOST, server.port)) - self.assertIn(s.cipher()[0], [ -- 'TLS13-AES-256-GCM-SHA384', -- 'TLS13-CHACHA20-POLY1305-SHA256', -- 'TLS13-AES-128-GCM-SHA256', -+ 'TLS_AES_256_GCM_SHA384', -+ 'TLS_CHACHA20_POLY1305_SHA256', -+ 'TLS_AES_128_GCM_SHA256', - ]) - - @unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL") -@@ -3391,22 +3395,19 @@ else: - client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) - client_context.verify_mode = ssl.CERT_REQUIRED - client_context.load_verify_locations(SIGNING_CA) -- if ssl.OPENSSL_VERSION_INFO >= (1, 0, 2): -- client_context.set_ciphers("AES128:AES256") -- server_context.set_ciphers("AES256") -- alg1 = "AES256" -- alg2 = "AES-256" -- else: -- client_context.set_ciphers("AES:3DES") -- server_context.set_ciphers("3DES") -- alg1 = "3DES" -- alg2 = "DES-CBC3" -+ client_context.set_ciphers("AES128:AES256") -+ server_context.set_ciphers("AES256") -+ expected_algs = [ -+ "AES256", "AES-256", -+ # TLS 1.3 ciphers are always enabled -+ "TLS_CHACHA20", "TLS_AES", -+ ] - - stats = server_params_test(client_context, server_context) - ciphers = stats['server_shared_ciphers'][0] - self.assertGreater(len(ciphers), 0) - for name, tls_version, bits in ciphers: -- if not alg1 in name.split("-") and alg2 not in name: -+ if not any (alg in name for alg in expected_algs): - self.fail(name) - - def test_read_write_after_close_raises_valuerror(self): --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch b/meta/recipes-devtools/python/python3/0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch deleted file mode 100644 index d6098472042..00000000000 --- a/meta/recipes-devtools/python/python3/0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch +++ /dev/null @@ -1,68 +0,0 @@ -From 7b40cb7293cb14e5c7c8ed123efaf9acb33edae2 Mon Sep 17 00:00:00 2001 -From: Christian Heimes <christian@python.org> -Date: Tue, 15 Aug 2017 10:33:43 +0200 -Subject: [PATCH] bpo-30714: ALPN changes for OpenSSL 1.1.0f (#2305) - -OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client -could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that. -The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN -callback can pretend to not been set. - -See https://github.com/openssl/openssl/pull/3158 for more details - -Signed-off-by: Christian Heimes <christian@python.org> - -Upstream-Status: Backport -[https://github.com/python/cpython/commit/7b40cb7293cb14e5c7c8ed123efaf9acb33edae2] - -Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> ---- - Doc/library/ssl.rst | 5 +++-- - Lib/test/test_ssl.py | 5 +++-- - .../next/Tests/2017-07-25-15-27-44.bpo-30715.Sp7bTF.rst | 2 ++ - 3 files changed, 8 insertions(+), 4 deletions(-) - create mode 100644 Misc/NEWS.d/next/Tests/2017-07-25-15-27-44.bpo-30715.Sp7bTF.rst - -diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst -index 729a239a1b..0a09e7e9d4 100644 ---- a/Doc/library/ssl.rst -+++ b/Doc/library/ssl.rst -@@ -1447,8 +1447,9 @@ to speed up repeated connections from the same clients. - This method will raise :exc:`NotImplementedError` if :data:`HAS_ALPN` is - False. - -- OpenSSL 1.1.0+ will abort the handshake and raise :exc:`SSLError` when -- both sides support ALPN but cannot agree on a protocol. -+ OpenSSL 1.1.0 to 1.1.0e will abort the handshake and raise :exc:`SSLError` -+ when both sides support ALPN but cannot agree on a protocol. 1.1.0f+ -+ behaves like 1.0.2, :meth:`SSLSocket.selected_alpn_protocol` returns None. - - .. versionadded:: 3.5 - -diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py -index d960d82065..104b7f377a 100644 ---- a/Lib/test/test_ssl.py -+++ b/Lib/test/test_ssl.py -@@ -3268,8 +3268,9 @@ if _have_threads: - except ssl.SSLError as e: - stats = e - -- if expected is None and IS_OPENSSL_1_1: -- # OpenSSL 1.1.0 raises handshake error -+ if (expected is None and IS_OPENSSL_1_1 -+ and ssl.OPENSSL_VERSION_INFO < (1, 1, 0, 6)): -+ # OpenSSL 1.1.0 to 1.1.0e raises handshake error - self.assertIsInstance(stats, ssl.SSLError) - else: - msg = "failed trying %s (s) and %s (c).\n" \ -diff --git a/Misc/NEWS.d/next/Tests/2017-07-25-15-27-44.bpo-30715.Sp7bTF.rst b/Misc/NEWS.d/next/Tests/2017-07-25-15-27-44.bpo-30715.Sp7bTF.rst -new file mode 100644 -index 0000000000..88394e585c ---- /dev/null -+++ b/Misc/NEWS.d/next/Tests/2017-07-25-15-27-44.bpo-30715.Sp7bTF.rst -@@ -0,0 +1,2 @@ -+Address ALPN callback changes for OpenSSL 1.1.0f. The latest version behaves -+like OpenSSL 1.0.2 and no longer aborts handshake. --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/03-fix-tkinter-detection.patch b/meta/recipes-devtools/python/python3/03-fix-tkinter-detection.patch deleted file mode 100644 index fddfd2b2f6a..00000000000 --- a/meta/recipes-devtools/python/python3/03-fix-tkinter-detection.patch +++ /dev/null @@ -1,42 +0,0 @@ -Upstream-Status: Inappropriate [embedded specific] - -# We need to supply STAGING_INCDIR here, otherwise the Tk headers -# will not be found. -# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille.de> - -Index: Python-3.3.0rc2/setup.py -=================================================================== ---- Python-3.3.0rc2.orig/setup.py 2012-09-20 15:24:14.009124003 -0700 -+++ Python-3.3.0rc2/setup.py 2012-09-20 15:25:08.449124963 -0700 -@@ -1620,7 +1620,7 @@ - dotversion = dotversion[:-1] + '.' + dotversion[-1] - tcl_include_sub = [] - tk_include_sub = [] -- for dir in inc_dirs: -+ for dir in [os.getenv("STAGING_INCDIR")]: - tcl_include_sub += [dir + os.sep + "tcl" + dotversion] - tk_include_sub += [dir + os.sep + "tk" + dotversion] - tk_include_sub += tcl_include_sub -@@ -1639,22 +1639,6 @@ - if dir not in include_dirs: - include_dirs.append(dir) - -- # Check for various platform-specific directories -- if host_platform == 'sunos5': -- include_dirs.append('/usr/openwin/include') -- added_lib_dirs.append('/usr/openwin/lib') -- elif os.path.exists('/usr/X11R6/include'): -- include_dirs.append('/usr/X11R6/include') -- added_lib_dirs.append('/usr/X11R6/lib64') -- added_lib_dirs.append('/usr/X11R6/lib') -- elif os.path.exists('/usr/X11R5/include'): -- include_dirs.append('/usr/X11R5/include') -- added_lib_dirs.append('/usr/X11R5/lib') -- else: -- # Assume default location for X11 -- include_dirs.append('/usr/X11/include') -- added_lib_dirs.append('/usr/X11/lib') -- - # If Cygwin, then verify that X is installed before proceeding - if host_platform == 'cygwin': - x11_inc = find_file('X11/Xlib.h', [], include_dirs) diff --git a/meta/recipes-devtools/python/python3/030-fixup-include-dirs.patch b/meta/recipes-devtools/python/python3/030-fixup-include-dirs.patch deleted file mode 100644 index 6beac7b8050..00000000000 --- a/meta/recipes-devtools/python/python3/030-fixup-include-dirs.patch +++ /dev/null @@ -1,33 +0,0 @@ -Do not hardcode /usr into include paths when cross compiling - --Khem - -Upstream-Status: Pending - ---- - setup.py | 15 ++------------- - 1 file changed, 2 insertions(+), 13 deletions(-) - -Index: Python-3.3.2/setup.py -=================================================================== ---- Python-3.3.2.orig/setup.py -+++ Python-3.3.2/setup.py -@@ -444,7 +444,8 @@ class PyBuildExt(build_ext): - # only change this for cross builds for 3.3, issues on Mageia - if cross_compiling: - self.add_gcc_paths() -- self.add_multiarch_paths() -+ if not cross_compiling: -+ self.add_multiarch_paths() - - # Add paths specified in the environment variables LDFLAGS and - # CPPFLAGS for header and library files. -@@ -480,7 +481,7 @@ class PyBuildExt(build_ext): - for directory in reversed(options.dirs): - add_dir_to_list(dir_list, directory) - -- if os.path.normpath(sys.base_prefix) != '/usr' \ -+ if not cross_compiling and os.path.normpath(sys.base_prefix) != '/usr' \ - and not sysconfig.get_config_var('PYTHONFRAMEWORK'): - # OSX note: Don't add LIBDIR and INCLUDEDIR to building a framework - # (PYTHONFRAMEWORK is set) to avoid # linking problems when diff --git a/meta/recipes-devtools/python/python3/080-distutils-dont_adjust_files.patch b/meta/recipes-devtools/python/python3/080-distutils-dont_adjust_files.patch deleted file mode 100644 index b96419a6387..00000000000 --- a/meta/recipes-devtools/python/python3/080-distutils-dont_adjust_files.patch +++ /dev/null @@ -1,92 +0,0 @@ -do not "adjust" python files before copying - --Khem - -Upstream-Status: Inappropriate [Embedded-Specific] - ---- - Lib/distutils/command/build_scripts.py | 43 +++------------------------------ - 1 file changed, 4 insertions(+), 39 deletions(-) - ---- a/Lib/distutils/command/build_scripts.py -+++ b/Lib/distutils/command/build_scripts.py -@@ -51,10 +51,7 @@ class build_scripts(Command): - - - def copy_scripts(self): -- """Copy each script listed in 'self.scripts'; if it's marked as a -- Python script in the Unix way (first line matches 'first_line_re', -- ie. starts with "\#!" and contains "python"), then adjust the first -- line to refer to the current Python interpreter as we copy. -+ """Copy each script listed in 'self.scripts' - """ - self.mkpath(self.build_dir) - outfiles = [] -@@ -78,64 +75,10 @@ class build_scripts(Command): - if not self.dry_run: - raise - f = None -- else: -- encoding, lines = tokenize.detect_encoding(f.readline) -- f.seek(0) -- first_line = f.readline() -- if not first_line: -- self.warn("%s is an empty file (skipping)" % script) -- continue -- -- match = first_line_re.match(first_line) -- if match: -- adjust = True -- post_interp = match.group(1) or b'' -- -- if adjust: -- log.info("copying and adjusting %s -> %s", script, -- self.build_dir) -- updated_files.append(outfile) -- if not self.dry_run: -- if not sysconfig.python_build: -- executable = self.executable -- else: -- executable = os.path.join( -- sysconfig.get_config_var("BINDIR"), -- "python%s%s" % (sysconfig.get_config_var("VERSION"), -- sysconfig.get_config_var("EXE"))) -- executable = os.fsencode(executable) -- shebang = b"#!" + executable + post_interp + b"\n" -- # Python parser starts to read a script using UTF-8 until -- # it gets a #coding:xxx cookie. The shebang has to be the -- # first line of a file, the #coding:xxx cookie cannot be -- # written before. So the shebang has to be decodable from -- # UTF-8. -- try: -- shebang.decode('utf-8') -- except UnicodeDecodeError: -- raise ValueError( -- "The shebang ({!r}) is not decodable " -- "from utf-8".format(shebang)) -- # If the script is encoded to a custom encoding (use a -- # #coding:xxx cookie), the shebang has to be decodable from -- # the script encoding too. -- try: -- shebang.decode(encoding) -- except UnicodeDecodeError: -- raise ValueError( -- "The shebang ({!r}) is not decodable " -- "from the script encoding ({})" -- .format(shebang, encoding)) -- with open(outfile, "wb") as outf: -- outf.write(shebang) -- outf.writelines(f.readlines()) -- if f: -- f.close() -- else: -- if f: -+ if f: - f.close() -- updated_files.append(outfile) -- self.copy_file(script, outfile) -+ updated_files.append(outfile) -+ self.copy_file(script, outfile) - - if os.name == 'posix': - for file in outfiles: diff --git a/meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch deleted file mode 100644 index c53ec0cfc85..00000000000 --- a/meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch +++ /dev/null @@ -1,57 +0,0 @@ -From 7630ab22578746d3d790d0598c0d279cf7afed97 Mon Sep 17 00:00:00 2001 -From: Khem Raj <raj.khem@gmail.com> -Date: Tue, 14 May 2013 15:00:26 -0700 -Subject: [PATCH 01/20] python3: Add target and native recipes - -Upstream-Status: Inappropriate [embedded specific] - -02/2015 Rebased for Python 3.4.2 - -# The proper prefix is inside our staging area. -# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de> -# Signed-off-by: Phil Blundell <philb@gnu.org> -# Signed-off-by: Khem Raj <raj.khem@gmail.com> -# Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> - ---- - Lib/distutils/sysconfig.py | 10 ++++++++-- - 1 file changed, 8 insertions(+), 2 deletions(-) - -diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py -index 573724d..390c485 100644 ---- a/Lib/distutils/sysconfig.py -+++ b/Lib/distutils/sysconfig.py -@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): - If 'prefix' is supplied, use it instead of sys.base_prefix or - sys.base_exec_prefix -- i.e., ignore 'plat_specific'. - """ -- if prefix is None: -+ if prefix is None and os.environ['STAGING_INCDIR'] != "": -+ prefix = os.environ['STAGING_INCDIR'].rstrip('include') -+ elif prefix is None: - prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX - if os.name == "posix": - if python_build: -@@ -125,6 +127,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): - If 'prefix' is supplied, use it instead of sys.base_prefix or - sys.base_exec_prefix -- i.e., ignore 'plat_specific'. - """ -+ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] -+ if prefix is None and os.environ['STAGING_LIBDIR'] != "": -+ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) -+ - if prefix is None: - if standard_lib: - prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX -@@ -133,7 +139,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): - - if os.name == "posix": - libpython = os.path.join(prefix, -- "lib", "python" + get_python_version()) -+ lib_basename, "python" + get_python_version()) - if standard_lib: - return libpython - else: --- -2.8.0.rc3 - diff --git a/meta/recipes-devtools/python/python3/130-readline-setup.patch b/meta/recipes-devtools/python/python3/130-readline-setup.patch deleted file mode 100644 index cfa712678b9..00000000000 --- a/meta/recipes-devtools/python/python3/130-readline-setup.patch +++ /dev/null @@ -1,55 +0,0 @@ -package python-readline - --Khem - -Upstream-Status: Inappropriate [Embedded Specific] - ---- a/setup.py -+++ b/setup.py -@@ -666,45 +666,7 @@ class PyBuildExt(build_ext): - # readline - do_readline = self.compiler.find_library_file(lib_dirs, 'readline') - readline_termcap_library = "" -- curses_library = "" -- # Cannot use os.popen here in py3k. -- tmpfile = os.path.join(self.build_temp, 'readline_termcap_lib') -- if not os.path.exists(self.build_temp): -- os.makedirs(self.build_temp) -- # Determine if readline is already linked against curses or tinfo. -- if do_readline: -- if cross_compiling: -- ret = os.system("%s -d %s | grep '(NEEDED)' > %s" \ -- % (sysconfig.get_config_var('READELF'), -- do_readline, tmpfile)) -- elif find_executable('ldd'): -- ret = os.system("ldd %s > %s" % (do_readline, tmpfile)) -- else: -- ret = 256 -- if ret >> 8 == 0: -- with open(tmpfile) as fp: -- for ln in fp: -- if 'curses' in ln: -- readline_termcap_library = re.sub( -- r'.*lib(n?cursesw?)\.so.*', r'\1', ln -- ).rstrip() -- break -- # termcap interface split out from ncurses -- if 'tinfo' in ln: -- readline_termcap_library = 'tinfo' -- break -- if os.path.exists(tmpfile): -- os.unlink(tmpfile) -- # Issue 7384: If readline is already linked against curses, -- # use the same library for the readline and curses modules. -- if 'curses' in readline_termcap_library: -- curses_library = readline_termcap_library -- elif self.compiler.find_library_file(lib_dirs, 'ncursesw'): -- curses_library = 'ncursesw' -- elif self.compiler.find_library_file(lib_dirs, 'ncurses'): -- curses_library = 'ncurses' -- elif self.compiler.find_library_file(lib_dirs, 'curses'): -- curses_library = 'curses' -+ curses_library = "ncursesw" - - if host_platform == 'darwin': - os_release = int(os.uname()[2].split('.')[0]) diff --git a/meta/recipes-devtools/python/python3/150-fix-setupterm.patch b/meta/recipes-devtools/python/python3/150-fix-setupterm.patch deleted file mode 100644 index 78d7c786701..00000000000 --- a/meta/recipes-devtools/python/python3/150-fix-setupterm.patch +++ /dev/null @@ -1,17 +0,0 @@ -Fix warning with newer compiler - --Khem - -Upstream-Status: Pending - ---- a/Modules/_cursesmodule.c -+++ b/Modules/_cursesmodule.c -@@ -116,7 +116,7 @@ char *PyCursesVersion = "2.2"; - #defines many common symbols (such as "lines") which breaks the - curses module in other ways. So the code will just specify - explicit prototypes here. */ --extern int setupterm(char *,int,int *); -+//extern int setupterm(char *,int,int *); - #ifdef __sgi - #include <term.h> - #endif diff --git a/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch b/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch deleted file mode 100644 index ae74ef531d3..00000000000 --- a/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch +++ /dev/null @@ -1,56 +0,0 @@ -From 7fd121bb7d6c25c2e0a1c31cf76fb9bd4a9794de Mon Sep 17 00:00:00 2001 -From: Markus Lehtonen <markus.lehtonen@linux.intel.com> -Date: Tue, 14 Aug 2018 14:11:35 +0800 -Subject: [PATCH 1/2] Use correct CFLAGS for extensions when cross-compiling - -Take PY_CFLAGS_NODIST into account, like in native build. This is needed -in order to to profile-optimized build. Also, pass EXTRA_CFLAGS to -profile-optimized build. - -Upstream-Status: Pending - -Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> ---- - Makefile.pre.in | 4 ++-- - setup.py | 3 ++- - 2 files changed, 4 insertions(+), 3 deletions(-) - -diff --git a/Makefile.pre.in b/Makefile.pre.in -index e2d5d3d..84bc3ff 100644 ---- a/Makefile.pre.in -+++ b/Makefile.pre.in -@@ -478,7 +478,7 @@ profile-opt: - $(MAKE) profile-removal - - build_all_generate_profile: -- $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" -+ $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" - - run_profile_task: - : # FIXME: can't run for a cross build -@@ -488,7 +488,7 @@ build_all_merge_profile: - $(LLVM_PROF_MERGER) - - build_all_use_profile: -- $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" -+ $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" - - # Compile and run with gcov - .PHONY=coverage coverage-lcov coverage-report -diff --git a/setup.py b/setup.py -index add3346..65e83b1 100644 ---- a/setup.py -+++ b/setup.py -@@ -263,7 +263,8 @@ class PyBuildExt(build_ext): - # compilers - if compiler is not None: - if cross_compiling: -- (ccshared,cflags) = (os.environ.get('CCSHARED') or '', os.environ.get('CFLAGS') or '') -+ (ccshared,cflags) = (os.environ.get('CCSHARED') or '', -+ (os.environ.get('CFLAGS') or '') + ' ' + sysconfig.get_config_var('PY_CFLAGS_NODIST')) - else: - (ccshared,cflags) = sysconfig.get_config_vars('CCSHARED','CFLAGS') - args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags --- -2.17.1 - diff --git a/meta/recipes-devtools/python/python3/avoid-ncursesw-include-path.patch b/meta/recipes-devtools/python/python3/avoid-ncursesw-include-path.patch deleted file mode 100644 index f29ae533717..00000000000 --- a/meta/recipes-devtools/python/python3/avoid-ncursesw-include-path.patch +++ /dev/null @@ -1,21 +0,0 @@ -Upstream-Status: Pending - -We should make sure that sysroot is used by gcc instead of assuming -hardcoded locations for include paths - - -Signed-off-by: Khem Raj <raj.khem@gmail.com> - -Index: Python-3.4.2/configure.ac -=================================================================== ---- Python-3.4.2.orig/configure.ac -+++ Python-3.4.2/configure.ac -@@ -4434,7 +4434,7 @@ fi - - # first curses header check - ac_save_cppflags="$CPPFLAGS" --CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -+CPPFLAGS="$CPPFLAGS -I=/usr/include/ncursesw" - - AC_CHECK_HEADERS(curses.h ncurses.h) - diff --git a/meta/recipes-devtools/python/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python/python3/avoid_warning_about_tkinter.patch deleted file mode 100644 index a3cc48c9a4d..00000000000 --- a/meta/recipes-devtools/python/python3/avoid_warning_about_tkinter.patch +++ /dev/null @@ -1,29 +0,0 @@ -_tkinter module needs tk module along with tcl. tk is not yet integrated -in yocto so we skip the check for this module. -Avoid a warning by not adding this module to missing variable. - -Upstream-Status: Inappropriate [distribution] - -Also simply disable the tk module since its not in DEPENDS. -Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> - -Index: Python-3.5.3/setup.py -=================================================================== ---- Python-3.5.3.orig/setup.py -+++ Python-3.5.3/setup.py -@@ -1558,10 +1558,12 @@ class PyBuildExt(build_ext): - self.extensions.extend(exts) - - # Call the method for detecting whether _tkinter can be compiled -- self.detect_tkinter(inc_dirs, lib_dirs) -+ # self.detect_tkinter(inc_dirs, lib_dirs) - -- if '_tkinter' not in [e.name for e in self.extensions]: -- missing.append('_tkinter') -+ # tkinter module will not be avalaible as yocto -+ # doesn't have tk integrated (yet) -+ #if '_tkinter' not in [e.name for e in self.extensions]: -+ # missing.append('_tkinter') - - ## # Uncomment these lines if you want to play with xxmodule.c - ## ext = Extension('xx', ['xxmodule.c']) diff --git a/meta/recipes-devtools/python/python3/cgi_py.patch b/meta/recipes-devtools/python/python3/cgi_py.patch deleted file mode 100644 index de504f9dcff..00000000000 --- a/meta/recipes-devtools/python/python3/cgi_py.patch +++ /dev/null @@ -1,23 +0,0 @@ -Lib/cgi.py: Update the script as mentioned in the comment - -Upstream-Status: Inappropriate [distribution] - -Signed-off-by: Mark Hatle <mark.hatle@windriver.com> - ---- Python-2.6.6/Lib/cgi.py.orig 2010-08-01 22:14:27.000000000 -0500 -+++ Python-2.6.6/Lib/cgi.py 2011-09-21 15:28:40.478208631 -0500 -@@ -1,13 +1,4 @@ --#! /usr/local/bin/python -- --# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is --# intentionally NOT "/usr/bin/env python". On many systems --# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI --# scripts, and /usr/local/bin is the default directory where Python is --# installed, so /usr/bin/env would be unable to find python. Granted, --# binary installations by Linux vendors often install Python in --# /usr/bin. So let those vendors patch cgi.py to match their choice --# of installation. -+#! /usr/bin/env python - - """Support module for CGI (Common Gateway Interface) scripts. - diff --git a/meta/recipes-devtools/python/python3/configure.ac-fix-LIBPL.patch b/meta/recipes-devtools/python/python3/configure.ac-fix-LIBPL.patch deleted file mode 100644 index 97214f9aa48..00000000000 --- a/meta/recipes-devtools/python/python3/configure.ac-fix-LIBPL.patch +++ /dev/null @@ -1,30 +0,0 @@ -From 7c4f8d87473d6238c120ec6031b58f83a17a39a5 Mon Sep 17 00:00:00 2001 -From: Robert Yang <liezhi.yang@windriver.com> -Date: Mon, 28 Dec 2015 22:52:06 -0800 -Subject: [PATCH] configure.ac: fix LIBPL - -Use LIBDIR rather than prefix/lib, so that it would work when lib64. - -Upstream-Status: Pending - -Signed-off-by: Robert Yang <liezhi.yang@windriver.com> ---- - configure.ac | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/configure.ac b/configure.ac -index 63aef8e..aefb27f 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -4366,7 +4366,7 @@ AC_MSG_RESULT($LDVERSION) - - dnl define LIBPL after ABIFLAGS and LDVERSION is defined. - AC_SUBST(PY_ENABLE_SHARED) --LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}" -+LIBPL='$(LIBDIR)'"/python${VERSION}/config-${LDVERSION}" - AC_SUBST(LIBPL) - - # Check whether right shifting a negative integer extends the sign bit --- -1.7.9.5 - diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py b/meta/recipes-devtools/python/python3/create_manifest3.py deleted file mode 100644 index 4da02a2991a..00000000000 --- a/meta/recipes-devtools/python/python3/create_manifest3.py +++ /dev/null @@ -1,433 +0,0 @@ -# This script is used as a bitbake task to create a new python manifest -# $ bitbake python -c create_manifest -# -# Our goal is to keep python-core as small as posible and add other python -# packages only when the user needs them, hence why we split upstream python -# into several packages. -# -# In a very simplistic way what this does is: -# Launch python and see specifically what is required for it to run at a minimum -# -# Go through the python-manifest file and launch a separate task for every single -# one of the files on each package, this task will check what was required for that -# specific module to run, these modules will be called dependencies. -# The output of such task will be a list of the modules or dependencies that were -# found for that file. -# -# Such output will be parsed by this script, we will look for each dependency on the -# manifest and if we find that another package already includes it, then we will add -# that package as an RDEPENDS to the package we are currently checking; in case we dont -# find the current dependency on any other package we will add it to the current package -# as part of FILES. -# -# -# This way we will create a new manifest from the data structure that was built during -# this process, on this new manifest each package will contain specifically only -# what it needs to run. -# -# There are some caveats which we try to deal with, such as repeated files on different -# packages, packages that include folders, wildcards, and special packages. -# Its also important to note that this method only works for python files, and shared -# libraries. Static libraries, header files and binaries need to be dealt with manually. -# -# This script differs from its python2 version mostly on how shared libraries are handled -# The manifest file for python3 has an extra field which contains the cached files for -# each package. -# Tha method to handle cached files does not work when a module includes a folder which -# itself contains the pycache folder, gladly this is almost never the case. -# -# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> - - -import sys -import subprocess -import json -import os -import collections - -# Get python version from ${PYTHON_MAJMIN} -pyversion = str(sys.argv[1]) - -# Hack to get native python search path (for folders), not fond of it but it works for now -pivot = 'recipe-sysroot-native' -for p in sys.path: - if pivot in p: - nativelibfolder = p[:p.find(pivot)+len(pivot)] - -# Empty dict to hold the whole manifest -new_manifest = collections.OrderedDict() - -# Check for repeated files, folders and wildcards -allfiles = [] -repeated = [] -wildcards = [] - -hasfolders = [] -allfolders = [] - -def isFolder(value): - value = value.replace('${PYTHON_MAJMIN}',pyversion) - if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): - return True - else: - return False - -def isCached(item): - if '__pycache__' in item: - return True - else: - return False - -def prepend_comments(comments, json_manifest): - with open(json_manifest, 'r+') as manifest: - json_contents = manifest.read() - manifest.seek(0, 0) - manifest.write(comments + json_contents) - -# Read existing JSON manifest -with open('python3-manifest.json') as manifest: - # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker - manifest_str = manifest.read() - json_start = manifest_str.find('# EOC') + 6 # EOC + \n - manifest.seek(0) - comments = manifest.read(json_start) - manifest_str = manifest.read() - old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) - -# -# First pass to get core-package functionality, because we base everything on the fact that core is actually working -# Not exactly the same so it should not be a function -# - -print ('Getting dependencies for package: core') - - -# This special call gets the core dependencies and -# appends to the old manifest so it doesnt hurt what it -# currently holds. -# This way when other packages check for dependencies -# on the new core package, they will still find them -# even when checking the old_manifest - -output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') -for coredep in output.split(): - coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') - if isCached(coredep): - if coredep not in old_manifest['core']['cached']: - old_manifest['core']['cached'].append(coredep) - else: - if coredep not in old_manifest['core']['files']: - old_manifest['core']['files'].append(coredep) - - -# The second step is to loop through the existing files contained in the core package -# according to the old manifest, identify if they are modules, or some other type -# of file that we cant import (directories, binaries, configs) in which case we -# can only assume they were added correctly (manually) so we ignore those and -# pass them to the manifest directly. - -for filedep in old_manifest['core']['files']: - if isFolder(filedep): - if isCached(filedep): - if filedep not in old_manifest['core']['cached']: - old_manifest['core']['cached'].append(filedep) - else: - if filedep not in old_manifest['core']['files']: - old_manifest['core']['files'].append(filedep) - continue - if '${bindir}' in filedep: - if filedep not in old_manifest['core']['files']: - old_manifest['core']['files'].append(filedep) - continue - if filedep == '': - continue - if '${includedir}' in filedep: - if filedep not in old_manifest['core']['files']: - old_manifest['core']['files'].append(filedep) - continue - - # Get actual module name , shouldnt be affected by libdir/bindir, etc. - pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] - - - # We now know that were dealing with a python module, so we can import it - # and check what its dependencies are. - # We launch a separate task for each module for deterministic behavior. - # Each module will only import what is necessary for it to work in specific. - # The output of each task will contain each module's dependencies - - print ('Getting dependencies for module: %s' % pymodule) - output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') - print ('The following dependencies were found for module %s:\n' % pymodule) - print (output) - - - for pymodule_dep in output.split(): - pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') - - if isCached(pymodule_dep): - if pymodule_dep not in old_manifest['core']['cached']: - old_manifest['core']['cached'].append(pymodule_dep) - else: - if pymodule_dep not in old_manifest['core']['files']: - old_manifest['core']['files'].append(pymodule_dep) - - -# At this point we are done with the core package. -# The old_manifest dictionary is updated only for the core package because -# all others will use this a base. - - -# To improve the script speed, we check which packages contain directories -# since we will be looping through (only) those later. -for pypkg in old_manifest: - for filedep in old_manifest[pypkg]['files']: - if isFolder(filedep): - print ('%s is a folder' % filedep) - if pypkg not in hasfolders: - hasfolders.append(pypkg) - if filedep not in allfolders: - allfolders.append(filedep) - - - -# This is the main loop that will handle each package. -# It works in a similar fashion than the step before, but -# we will now be updating a new dictionary that will eventually -# become the new manifest. -# -# The following loops though all packages in the manifest, -# through all files on each of them, and checks whether or not -# they are modules and can be imported. -# If they can be imported, then it checks for dependencies for -# each of them by launching a separate task. -# The output of that task is then parsed and the manifest is updated -# accordingly, wether it should add the module on FILES for the current package -# or if that module already belongs to another package then the current one -# will RDEPEND on it - -for pypkg in old_manifest: - # Use an empty dict as data structure to hold data for each package and fill it up - new_manifest[pypkg] = collections.OrderedDict() - new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] - new_manifest[pypkg]['rdepends'] = [] - new_manifest[pypkg]['files'] = [] - new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] - - # All packages should depend on core - if pypkg != 'core': - new_manifest[pypkg]['rdepends'].append('core') - new_manifest[pypkg]['cached'] = [] - - print('\n') - print('--------------------------') - print ('Handling package %s' % pypkg) - print('--------------------------') - - # Handle special cases, we assume that when they were manually added - # to the manifest we knew what we were doing. - special_packages = ['misc', 'modules', 'dev', 'tests'] - if pypkg in special_packages or 'staticdev' in pypkg: - print('Passing %s package directly' % pypkg) - new_manifest[pypkg] = old_manifest[pypkg] - continue - - for filedep in old_manifest[pypkg]['files']: - # We already handled core on the first pass, we can ignore it now - if pypkg == 'core': - if filedep not in new_manifest[pypkg]['files']: - new_manifest[pypkg]['files'].append(filedep) - continue - - # Handle/ignore what we cant import - if isFolder(filedep): - new_manifest[pypkg]['files'].append(filedep) - # Asyncio (and others) are both the package and the folder name, we should not skip those... - path,mod = os.path.split(filedep) - if mod != pypkg: - continue - if '${bindir}' in filedep: - if filedep not in new_manifest[pypkg]['files']: - new_manifest[pypkg]['files'].append(filedep) - continue - if filedep == '': - continue - if '${includedir}' in filedep: - if filedep not in new_manifest[pypkg]['files']: - new_manifest[pypkg]['files'].append(filedep) - continue - - # Get actual module name , shouldnt be affected by libdir/bindir, etc. - # We need to check if the imported module comes from another (e.g. sqlite3.dump) - path,pymodule = os.path.split(filedep) - path = os.path.basename(path) - pymodule = os.path.splitext(os.path.basename(pymodule))[0] - - # If this condition is met, it means we need to import it from another module - # or its the folder itself (e.g. unittest) - if path == pypkg: - if pymodule: - pymodule = path + '.' + pymodule - else: - pymodule = path - - - - # We now know that were dealing with a python module, so we can import it - # and check what its dependencies are. - # We launch a separate task for each module for deterministic behavior. - # Each module will only import what is necessary for it to work in specific. - # The output of each task will contain each module's dependencies - - print ('\nGetting dependencies for module: %s' % pymodule) - output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') - print ('The following dependencies were found for module %s:\n' % pymodule) - print (output) - - reportFILES = [] - reportRDEPS = [] - - for pymodule_dep in output.split(): - - # Warning: This first part is ugly - # One of the dependencies that was found, could be inside of one of the folders included by another package - # We need to check if this happens so we can add the package containing the folder as an rdependency - # e.g. Folder encodings contained in codecs - # This would be solved if no packages included any folders - - # This can be done in two ways: - # 1 - We assume that if we take out the filename from the path we would get - # the folder string, then we would check if folder string is in the list of folders - # This would not work if a package contains a folder which contains another folder - # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 - # folder_string would not match any value contained in the list of folders - # - # 2 - We do it the other way around, checking if the folder is contained in the path - # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 - # is folder_string inside path/folder1/folder2/filename?, - # Yes, it works, but we waste a couple of milliseconds. - - pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') - inFolders = False - for folder in allfolders: - # The module could have a directory named after it, e.g. xml, if we take out the filename from the path - # we'll end up with ${libdir}, and we want ${libdir}/xml - if isFolder(pymodule_dep): - check_path = pymodule_dep - else: - check_path = os.path.dirname(pymodule_dep) - if folder in check_path : - inFolders = True # Did we find a folder? - folderFound = False # Second flag to break inner for - # Loop only through packages which contain folders - for pypkg_with_folder in hasfolders: - if (folderFound == False): - # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) - for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: - if folder_dep == folder: - print ('%s folder found in %s' % (folder, pypkg_with_folder)) - folderFound = True - if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: - new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) - else: - break - - # A folder was found so we're done with this item, we can go on - if inFolders: - continue - - - - # No directories beyond this point - # We might already have this module on the dictionary since it could depend on a (previously checked) module - if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: - # Handle core as a special package, we already did it so we pass it to NEW data structure directly - if pypkg == 'core': - print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) - if pymodule_dep.endswith('*'): - wildcards.append(pymodule_dep) - if isCached(pymodule_dep): - new_manifest[pypkg]['cached'].append(pymodule_dep) - else: - new_manifest[pypkg]['files'].append(pymodule_dep) - - # Check for repeated files - if pymodule_dep not in allfiles: - allfiles.append(pymodule_dep) - else: - if pymodule_dep not in repeated: - repeated.append(pymodule_dep) - else: - - - # Last step: Figure out if we this belongs to FILES or RDEPENDS - # We check if this module is already contained on another package, so we add that one - # as an RDEPENDS, or if its not, it means it should be contained on the current - # package, and we should add it to FILES - for possible_rdep in old_manifest: - # Debug - # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) - if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: - # Since were nesting, we need to check its not the same pypkg - if(possible_rdep != pypkg): - if possible_rdep not in new_manifest[pypkg]['rdepends']: - # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs - reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) - new_manifest[pypkg]['rdepends'].append(possible_rdep) - break - else: - - # Since this module wasnt found on another package, it is not an RDEP, - # so we add it to FILES for this package. - # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) - if os.path.basename(pymodule_dep) != pypkg: - reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) - if isCached(pymodule_dep): - new_manifest[pypkg]['cached'].append(pymodule_dep) - else: - new_manifest[pypkg]['files'].append(pymodule_dep) - if pymodule_dep.endswith('*'): - wildcards.append(pymodule_dep) - if pymodule_dep not in allfiles: - allfiles.append(pymodule_dep) - else: - if pymodule_dep not in repeated: - repeated.append(pymodule_dep) - - print('\n') - print('#################################') - print('Summary for module %s' % pymodule) - print('FILES found for module %s:' % pymodule) - print(''.join(reportFILES)) - print('RDEPENDS found for module %s:' % pymodule) - print(''.join(reportRDEPS)) - print('#################################') - -print('The following FILES contain wildcards, please check if they are necessary') -print(wildcards) -print('The following FILES contain folders, please check if they are necessary') -print(hasfolders) - - -# Sort it just so it looks nicer -for pypkg in new_manifest: - new_manifest[pypkg]['files'].sort() - new_manifest[pypkg]['cached'].sort() - new_manifest[pypkg]['rdepends'].sort() - -# Create the manifest from the data structure that was built -with open('python3-manifest.json.new','w') as outfile: - json.dump(new_manifest,outfile, indent=4) - outfile.write('\n') - -prepend_comments(comments,'python3-manifest.json.new') - -if (repeated): - error_msg = '\n\nERROR:\n' - error_msg += 'The following files are repeated (contained in more than one package),\n' - error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' - error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' - error_msg += '\n'.join(repeated) - error_msg += '\n' - sys.exit(error_msg) - diff --git a/meta/recipes-devtools/python/python3/fix_for_using_different_libdir.patch b/meta/recipes-devtools/python/python3/fix_for_using_different_libdir.patch deleted file mode 100644 index 0610565d3c1..00000000000 --- a/meta/recipes-devtools/python/python3/fix_for_using_different_libdir.patch +++ /dev/null @@ -1,54 +0,0 @@ -Upstream-Status: Inappropriate [Embedded specific] - -This patch fixes issuing with different libdir like lib64. -This patch makes the native python binary modules findable -in the install process of the host python. - -Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com> -Date: 2012/03/14 - -Updated for python 2.7.3 -Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com> -Date: 2012/05/01 - -Index: Python-3.3.0rc2/Lib/sysconfig.py -=================================================================== ---- Python-3.3.0rc2.orig/Lib/sysconfig.py 2012-09-20 22:50:11.000000000 -0700 -+++ Python-3.3.0rc2/Lib/sysconfig.py 2012-09-20 22:53:01.561123396 -0700 -@@ -21,9 +21,9 @@ - - _INSTALL_SCHEMES = { - 'posix_prefix': { -- 'stdlib': '{installed_base}/'+sys.lib+'/python{py_version_short}', -+ 'stdlib': '{base}/'+sys.lib+'/python{py_version_short}', - 'platstdlib': '{platbase}/'+sys.lib+'/python{py_version_short}', -- 'purelib': '{base}/lib/python{py_version_short}/site-packages', -+ 'purelib': '{base}/'+sys.lib+'/python{py_version_short}/site-packages', - 'platlib': '{platbase}/'+sys.lib+'/python{py_version_short}/site-packages', - 'include': - '{installed_base}/include/python{py_version_short}{abiflags}', -@@ -83,7 +83,7 @@ - 'posix_user': { - 'stdlib': '{userbase}/'+sys.lib+'/python{py_version_short}', - 'platstdlib': '{userbase}/'+sys.lib+'/python{py_version_short}', -- 'purelib': '{userbase}/lib/python{py_version_short}/site-packages', -+ 'purelib': '{userbase}/'+sys.lib+'/python{py_version_short}/site-packages', - 'platlib': '{userbase}/'+sys.lib+'/python{py_version_short}/site-packages', - 'include': '{userbase}/include/python{py_version_short}', - 'scripts': '{userbase}/bin', -Index: Python-3.3.0rc2/Makefile.pre.in -=================================================================== ---- Python-3.3.0rc2.orig/Makefile.pre.in 2012-09-20 22:50:11.000000000 -0700 -+++ Python-3.3.0rc2/Makefile.pre.in 2012-09-20 22:50:54.245123997 -0700 -@@ -1080,9 +1080,9 @@ - $(INSTALL_DATA) $(srcdir)/Modules/xxmodule.c \ - $(DESTDIR)$(LIBDEST)/distutils/tests ; \ - fi -- -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ -+ -PYTHONPATH=$(DESTDIR)$(LIBDEST):${CROSSPYTHONPATH} $(RUNSHARED) \ - $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/Grammar.txt -- -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ -+ -PYTHONPATH=$(DESTDIR)$(LIBDEST):${CROSSPYTHONPATH} $(RUNSHARED) \ - $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt - - # Create the PLATDIR source directory, if one wasn't distributed.. diff --git a/meta/recipes-devtools/python/python3/float-endian.patch b/meta/recipes-devtools/python/python3/float-endian.patch deleted file mode 100644 index 6ba3f5c252d..00000000000 --- a/meta/recipes-devtools/python/python3/float-endian.patch +++ /dev/null @@ -1,212 +0,0 @@ -Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles, -and falls back onto "I don't know" if it can't run code. This results in -crippled floating point numbers in Python, and the regression tests fail. - -Instead of running code, take a macro from autoconf-archive which compiles C -with a special double in which has an ASCII representation, and then greps the -binary to identify the format. - -Upstream-Status: Submitted [https://bugs.python.org/issue34585] -Signed-off-by: Ross Burton <ross.burton@intel.com> - -From 50df2a4c3a65ed06322be7c26d42b06ce81730c1 Mon Sep 17 00:00:00 2001 -From: Ross Burton <ross.burton@intel.com> -Date: Wed, 5 Sep 2018 11:45:52 +0100 -Subject: [PATCH] Don't do runtime test to get float byte order - ---- - configure.ac | 74 +++++------------------------------ - m4/ax_c_float_words_bigendian.m4 | 83 ++++++++++++++++++++++++++++++++++++++++ - 2 files changed, 92 insertions(+), 65 deletions(-) - create mode 100644 m4/ax_c_float_words_bigendian.m4 - -diff --git a/configure.ac b/configure.ac -index c9b755f0f4..1215969871 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -9,6 +9,8 @@ AC_PREREQ(2.65) - - AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) - -+AC_CONFIG_MACRO_DIR(m4) -+ - AC_SUBST(BASECPPFLAGS) - if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then - # If we're building out-of-tree, we need to make sure the following -@@ -4128,77 +4130,19 @@ fi - # * Check for various properties of floating point * - # ************************************************** - --AC_MSG_CHECKING(whether C doubles are little-endian IEEE 754 binary64) --AC_CACHE_VAL(ac_cv_little_endian_double, [ --AC_RUN_IFELSE([AC_LANG_SOURCE([[ --#include <string.h> --int main() { -- double x = 9006104071832581.0; -- if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) -- return 0; -- else -- return 1; --} --]])], --[ac_cv_little_endian_double=yes], --[ac_cv_little_endian_double=no], --[ac_cv_little_endian_double=no])]) --AC_MSG_RESULT($ac_cv_little_endian_double) --if test "$ac_cv_little_endian_double" = yes --then -- AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, -- [Define if C doubles are 64-bit IEEE 754 binary format, stored -- with the least significant byte first]) --fi -- --AC_MSG_CHECKING(whether C doubles are big-endian IEEE 754 binary64) --AC_CACHE_VAL(ac_cv_big_endian_double, [ --AC_RUN_IFELSE([AC_LANG_SOURCE([[ --#include <string.h> --int main() { -- double x = 9006104071832581.0; -- if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) -- return 0; -- else -- return 1; --} --]])], --[ac_cv_big_endian_double=yes], --[ac_cv_big_endian_double=no], --[ac_cv_big_endian_double=no])]) --AC_MSG_RESULT($ac_cv_big_endian_double) --if test "$ac_cv_big_endian_double" = yes -+AX_C_FLOAT_WORDS_BIGENDIAN -+if test "$ax_cv_c_float_words_bigendian" = "yes" - then - AC_DEFINE(DOUBLE_IS_BIG_ENDIAN_IEEE754, 1, - [Define if C doubles are 64-bit IEEE 754 binary format, stored - with the most significant byte first]) --fi -- --# Some ARM platforms use a mixed-endian representation for doubles. --# While Python doesn't currently have full support for these platforms --# (see e.g., issue 1762561), we can at least make sure that float <-> string --# conversions work. --AC_MSG_CHECKING(whether C doubles are ARM mixed-endian IEEE 754 binary64) --AC_CACHE_VAL(ac_cv_mixed_endian_double, [ --AC_RUN_IFELSE([AC_LANG_SOURCE([[ --#include <string.h> --int main() { -- double x = 9006104071832581.0; -- if (memcmp(&x, "\x01\xff\x3f\x43\x05\x04\x03\x02", 8) == 0) -- return 0; -- else -- return 1; --} --]])], --[ac_cv_mixed_endian_double=yes], --[ac_cv_mixed_endian_double=no], --[ac_cv_mixed_endian_double=no])]) --AC_MSG_RESULT($ac_cv_mixed_endian_double) --if test "$ac_cv_mixed_endian_double" = yes -+elif test "$ax_cv_c_float_words_bigendian" = "no" - then -- AC_DEFINE(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754, 1, -+ AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, - [Define if C doubles are 64-bit IEEE 754 binary format, stored -- in ARM mixed-endian order (byte order 45670123)]) -+ with the least significant byte first]) -+else -+ AC_MSG_ERROR([Cannot identify floating point byte order]) - fi - - # The short float repr introduced in Python 3.1 requires the -diff --git a/m4/ax_c_float_words_bigendian.m4 b/m4/ax_c_float_words_bigendian.m4 -new file mode 100644 -index 0000000000..216b90d803 ---- /dev/null -+++ b/m4/ax_c_float_words_bigendian.m4 -@@ -0,0 +1,83 @@ -+# =============================================================================== -+# https://www.gnu.org/software/autoconf-archive/ax_c_float_words_bigendian.html -+# =============================================================================== -+# -+# SYNOPSIS -+# -+# AX_C_FLOAT_WORDS_BIGENDIAN([ACTION-IF-TRUE], [ACTION-IF-FALSE], [ACTION-IF-UNKNOWN]) -+# -+# DESCRIPTION -+# -+# Checks the ordering of words within a multi-word float. This check is -+# necessary because on some systems (e.g. certain ARM systems), the float -+# word ordering can be different from the byte ordering. In a multi-word -+# float context, "big-endian" implies that the word containing the sign -+# bit is found in the memory location with the lowest address. This -+# implementation was inspired by the AC_C_BIGENDIAN macro in autoconf. -+# -+# The endianness is detected by first compiling C code that contains a -+# special double float value, then grepping the resulting object file for -+# certain strings of ASCII values. The double is specially crafted to have -+# a binary representation that corresponds with a simple string. In this -+# implementation, the string "noonsees" was selected because the -+# individual word values ("noon" and "sees") are palindromes, thus making -+# this test byte-order agnostic. If grep finds the string "noonsees" in -+# the object file, the target platform stores float words in big-endian -+# order. If grep finds "seesnoon", float words are in little-endian order. -+# If neither value is found, the user is instructed to specify the -+# ordering. -+# -+# LICENSE -+# -+# Copyright (c) 2008 Daniel Amelang <dan@amelang.net> -+# -+# Copying and distribution of this file, with or without modification, are -+# permitted in any medium without royalty provided the copyright notice -+# and this notice are preserved. This file is offered as-is, without any -+# warranty. -+ -+#serial 11 -+ -+AC_DEFUN([AX_C_FLOAT_WORDS_BIGENDIAN], -+ [AC_CACHE_CHECK(whether float word ordering is bigendian, -+ ax_cv_c_float_words_bigendian, [ -+ -+ax_cv_c_float_words_bigendian=unknown -+AC_COMPILE_IFELSE([AC_LANG_SOURCE([[ -+ -+double d = 90904234967036810337470478905505011476211692735615632014797120844053488865816695273723469097858056257517020191247487429516932130503560650002327564517570778480236724525140520121371739201496540132640109977779420565776568942592.0; -+ -+]])], [ -+ -+if grep noonsees conftest.$ac_objext >/dev/null ; then -+ ax_cv_c_float_words_bigendian=yes -+fi -+if grep seesnoon conftest.$ac_objext >/dev/null ; then -+ if test "$ax_cv_c_float_words_bigendian" = unknown; then -+ ax_cv_c_float_words_bigendian=no -+ else -+ ax_cv_c_float_words_bigendian=unknown -+ fi -+fi -+ -+])]) -+ -+case $ax_cv_c_float_words_bigendian in -+ yes) -+ m4_default([$1], -+ [AC_DEFINE([FLOAT_WORDS_BIGENDIAN], 1, -+ [Define to 1 if your system stores words within floats -+ with the most significant word first])]) ;; -+ no) -+ $2 ;; -+ *) -+ m4_default([$3], -+ [AC_MSG_ERROR([ -+ -+Unknown float word ordering. You need to manually preset -+ax_cv_c_float_words_bigendian=no (or yes) according to your system. -+ -+ ])]) ;; -+esac -+ -+])# AX_C_FLOAT_WORDS_BIGENDIAN --- -2.11.0 - diff --git a/meta/recipes-devtools/python/python3/ftplib.patch b/meta/recipes-devtools/python/python3/ftplib.patch deleted file mode 100644 index 49c5b2736b9..00000000000 --- a/meta/recipes-devtools/python/python3/ftplib.patch +++ /dev/null @@ -1,60 +0,0 @@ -Upstream-Status: Backport -Signed-off-by: Ross Burton <ross.burton@intel.com> - -From cabe916dc694997d4892b58986e73a713d5a2f8d Mon Sep 17 00:00:00 2001 -From: "Miss Islington (bot)" - <31488909+miss-islington@users.noreply.github.com> -Date: Thu, 16 Aug 2018 15:38:03 -0400 -Subject: [PATCH] [3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) - (#8790) - -Read from data socket to avoid "[SSL] shutdown while in init" exception -during shutdown of the dummy server. - -Signed-off-by: Christian Heimes <christian@python.org> - - -<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) --> -https://bugs.python.org/issue34391 -<!-- /issue-number --> -(cherry picked from commit 1590c393360df059160145e7475754427bfc6680) - - -Co-authored-by: Christian Heimes <christian@python.org> ---- - Lib/test/test_ftplib.py | 5 +++++ - Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst | 1 + - 2 files changed, 6 insertions(+) - create mode 100644 Misc/NEWS.d/next/Tests/2018-08-16-18-48-47.bpo-34391.ouNfxC.rst - -diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py -index 44dd73aeca..4ff2f71afb 100644 ---- a/Lib/test/test_ftplib.py -+++ b/Lib/test/test_ftplib.py -@@ -876,18 +876,23 @@ class TestTLS_FTPClass(TestCase): - # clear text - with self.client.transfercmd('list') as sock: - self.assertNotIsInstance(sock, ssl.SSLSocket) -+ self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii')) - self.assertEqual(self.client.voidresp(), "226 transfer complete") - - # secured, after PROT P - self.client.prot_p() - with self.client.transfercmd('list') as sock: - self.assertIsInstance(sock, ssl.SSLSocket) -+ # consume from SSL socket to finalize handshake and avoid -+ # "SSLError [SSL] shutdown while in init" -+ self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii')) - self.assertEqual(self.client.voidresp(), "226 transfer complete") - - # PROT C is issued, the connection must be in cleartext again - self.client.prot_c() - with self.client.transfercmd('list') as sock: - self.assertNotIsInstance(sock, ssl.SSLSocket) -+ self.assertEqual(sock.recv(1024), LIST_DATA.encode('ascii')) - self.assertEqual(self.client.voidresp(), "226 transfer complete") - - def test_login(self): --- -2.11.0 - diff --git a/meta/recipes-devtools/python/python3/get_module_deps3.py b/meta/recipes-devtools/python/python3/get_module_deps3.py deleted file mode 100644 index fd12baad84e..00000000000 --- a/meta/recipes-devtools/python/python3/get_module_deps3.py +++ /dev/null @@ -1,146 +0,0 @@ -# This script is launched on separate task for each python module -# It checks for dependencies for that specific module and prints -# them out, the output of this execution will have all dependencies -# for a specific module, which will be parsed an dealt on create_manifest.py -# -# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com> - -# We can get a log per module, for all the dependencies that were found, but its messy. -debug=False - -import sys - -# We can get a list of the modules which are currently required to run python -# so we run python-core and get its modules, we then import what we need -# and check what modules are currently running, if we substract them from the -# modules we had initially, we get the dependencies for the module we imported. - -# We use importlib to achieve this, so we also need to know what modules importlib needs -import importlib - -core_deps=set(sys.modules) - -def fix_path(dep_path): - import os - # We DONT want the path on our HOST system - pivot='recipe-sysroot-native' - dep_path=dep_path[dep_path.find(pivot)+len(pivot):] - - if '/usr/bin' in dep_path: - dep_path = dep_path.replace('/usr/bin''${bindir}') - - # Handle multilib, is there a better way? - if '/usr/lib32' in dep_path: - dep_path = dep_path.replace('/usr/lib32','${libdir}') - if '/usr/lib64' in dep_path: - dep_path = dep_path.replace('/usr/lib64','${libdir}') - if '/usr/lib' in dep_path: - dep_path = dep_path.replace('/usr/lib','${libdir}') - if '/usr/include' in dep_path: - dep_path = dep_path.replace('/usr/include','${includedir}') - if '__init__.' in dep_path: - dep_path = os.path.split(dep_path)[0] - return dep_path - - -# Module to import was passed as an argument -current_module = str(sys.argv[1]).rstrip() -if(debug==True): - log = open('log_%s' % current_module,'w') - log.write('Module %s generated the following dependencies:\n' % current_module) -try: - importlib.import_module('%s' % current_module) -except ImportError as e: - if (debug==True): - log.write('Module was not found') - pass - - -# Get current module dependencies, dif will contain a list of specific deps for this module -module_deps=set(sys.modules) - -# We handle the core package (1st pass on create_manifest.py) as a special case -if current_module == 'python-core-package': - dif = core_deps -else: - # We know this is not the core package, so there must be a difference. - dif = module_deps-core_deps - - -# Check where each dependency came from -for item in dif: - dep_path='' - try: - if (debug==True): - log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') - dep_path = sys.modules['%s' % item].__file__ - except AttributeError as e: - # Deals with thread (builtin module) not having __file__ attribute - if debug==True: - log.write(item + ' ') - log.write(str(e)) - log.write('\n') - pass - except NameError as e: - # Deals with NameError: name 'dep_path' is not defined - # because module is not found (wasn't compiled?), e.g. bddsm - if (debug==True): - log.write(item+' ') - log.write(str(e)) - pass - - # Site-customize is a special case since we (OpenEmbedded) put it there manually - if 'sitecustomize' in dep_path: - dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' - # Prints out result, which is what will be used by create_manifest - print (dep_path) - continue - - dep_path = fix_path(dep_path) - - import sysconfig - soabi=sysconfig.get_config_var('SOABI') - # Check if its a shared library and deconstruct it - if soabi in dep_path: - if (debug==True): - log.write('Shared library found in %s' % dep_path) - dep_path = dep_path.replace(soabi,'*') - print (dep_path) - continue - - if (debug==True): - log.write(dep_path+'\n') - # Prints out result, which is what will be used by create_manifest - print (dep_path) - - - import imp - cpython_tag = imp.get_tag() - cached='' - # Theres no naive way to find *.pyc files on python3 - try: - if (debug==True): - log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') - cached = sys.modules['%s' % item].__cached__ - except AttributeError as e: - # Deals with thread (builtin module) not having __cached__ attribute - if debug==True: - log.write(item + ' ') - log.write(str(e)) - log.write('\n') - pass - except NameError as e: - # Deals with NameError: name 'cached' is not defined - if (debug==True): - log.write(item+' ') - log.write(str(e)) - pass - if cached is not None: - if (debug==True): - log.write(cached) - cached = fix_path(cached) - cached = cached.replace(cpython_tag,'*') - print (cached) - -if debug==True: - log.close() diff --git a/meta/recipes-devtools/python/python3/host_include_contamination.patch b/meta/recipes-devtools/python/python3/host_include_contamination.patch deleted file mode 100644 index ef2054d9a9e..00000000000 --- a/meta/recipes-devtools/python/python3/host_include_contamination.patch +++ /dev/null @@ -1,28 +0,0 @@ -when building python for qemux86-64 on ubuntu 11.10/64bit -it gropes into host includes and then mixes them with cross -includes and as a result some modules fail to compile and link -one of the modules is python-elementtree which is then not -found during image creation - -Proble is that setup.py tries to add native includes that newer -ubuntu has introduced for multiarch support. But that should -only happen for native builds and not cross building python -so we add a check here. - -Signed-off-by: Khem Raj <raj.khem@gmail.com> -Upstream-Status: Pending - -Index: Python-3.3.0rc2/setup.py -=================================================================== ---- Python-3.3.0rc2.orig/setup.py 2012-09-20 21:54:50.000000000 -0700 -+++ Python-3.3.0rc2/setup.py 2012-09-20 21:57:35.029123858 -0700 -@@ -402,6 +402,9 @@ - - if not find_executable('dpkg-architecture'): - return -+ if cross_compiling: -+ return -+ - opt = '' - if cross_compiling: - opt = '-t' + sysconfig.get_config_var('HOST_GNU_TYPE') diff --git a/meta/recipes-devtools/python/python3/pass-missing-libraries-to-Extension-for-mul.patch b/meta/recipes-devtools/python/python3/pass-missing-libraries-to-Extension-for-mul.patch deleted file mode 100644 index 5c3af6b6269..00000000000 --- a/meta/recipes-devtools/python/python3/pass-missing-libraries-to-Extension-for-mul.patch +++ /dev/null @@ -1,82 +0,0 @@ -From a784b70d47ba2104afbcfd805e2a66cdc2109ec5 Mon Sep 17 00:00:00 2001 -From: Hongxu Jia <hongxu.jia@windriver.com> -Date: Fri, 4 Aug 2017 11:16:14 +0800 -Subject: [PATCH] setup.py: pass missing libraries to Extension for multiprocessing module - -In the following commit: -... -commit e711cafab13efc9c1fe6c5cd75826401445eb585 -Author: Benjamin Peterson <benjamin@python.org> -Date: Wed Jun 11 16:44:04 2008 +0000 - - Merged revisions 64104,64117 via svnmerge from - svn+ssh://pythondev@svn.python.org/python/trunk -... -(see diff in setup.py) -It assigned libraries for multiprocessing module according -the host_platform, but not pass it to Extension. - -In glibc, the following commit caused two definition of -sem_getvalue are different. -https://sourceware.org/git/?p=glibc.git;a=commit;h=042e1521c794a945edc43b5bfa7e69ad70420524 -(see diff in nptl/sem_getvalue.c for detail) -`__new_sem_getvalue' is the latest sem_getvalue@@GLIBC_2.1 -and `__old_sem_getvalue' is to compat the old version -sem_getvalue@GLIBC_2.0. - -To build python for embedded Linux systems: -http://www.yoctoproject.org/docs/2.3.1/yocto-project-qs/yocto-project-qs.html -If not explicitly link to library pthread (-lpthread), it will -load glibc's sem_getvalue randomly at runtime. - -Such as build python on linux x86_64 host and run the python -on linux x86_32 target. If not link library pthread, it caused -multiprocessing bounded semaphore could not work correctly. -... ->>> import multiprocessing ->>> pool_sema = multiprocessing.BoundedSemaphore(value=1) ->>> pool_sema.acquire() -True ->>> pool_sema.release() -Traceback (most recent call last): - File "<stdin>", line 1, in <module> -ValueError: semaphore or lock released too many times -... - -And the semaphore issue also caused multiprocessing.Queue().put() hung. - -Upstream-Status: Submitted [https://github.com/python/cpython/pull/2999] - -Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> ---- - setup.py | 7 +++++-- - 1 file changed, 5 insertions(+), 2 deletions(-) - -diff --git a/setup.py b/setup.py -index 4f0f522..d05707d 100644 ---- a/setup.py -+++ b/setup.py -@@ -1606,8 +1606,10 @@ class PyBuildExt(build_ext): - elif host_platform.startswith('netbsd'): - macros = dict() - libraries = [] -- -- else: # Linux and other unices -+ elif host_platform.startswith(('linux')): -+ macros = dict() -+ libraries = ['pthread'] -+ else: # Other unices - macros = dict() - libraries = ['rt'] - -@@ -1626,6 +1628,7 @@ class PyBuildExt(build_ext): - if sysconfig.get_config_var('WITH_THREAD'): - exts.append ( Extension('_multiprocessing', multiprocessing_srcs, - define_macros=list(macros.items()), -+ libraries=libraries, - include_dirs=["Modules/_multiprocessing"])) - else: - missing.append('_multiprocessing') --- -2.7.4 - diff --git a/meta/recipes-devtools/python/python3/python-3.3-multilib.patch b/meta/recipes-devtools/python/python3/python-3.3-multilib.patch deleted file mode 100644 index cc35dc1617d..00000000000 --- a/meta/recipes-devtools/python/python3/python-3.3-multilib.patch +++ /dev/null @@ -1,363 +0,0 @@ -From 51fe6f22d0ba113674fb358bd11d75fe659bd26e Mon Sep 17 00:00:00 2001 -From: Khem Raj <raj.khem@gmail.com> -Date: Tue, 14 May 2013 15:00:26 -0700 -Subject: [PATCH 01/13] get the sys.lib from python itself and do not use - hardcoded value of 'lib' - -02/2015 Rebased for 3.4.2 - -Upstream-Status: Pending -Signed-off-by: Khem Raj <raj.khem@gmail.com> -Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> - ---- - Include/pythonrun.h | 3 +++ - Lib/distutils/command/install.py | 4 +++- - Lib/pydoc.py | 2 +- - Lib/site.py | 4 ++-- - Lib/sysconfig.py | 18 +++++++++--------- - Lib/trace.py | 4 ++-- - Makefile.pre.in | 7 +++++-- - Modules/getpath.c | 10 +++++++++- - Python/getplatform.c | 20 ++++++++++++++++++++ - Python/sysmodule.c | 4 ++++ - configure.ac | 35 +++++++++++++++++++++++++++++++++++ - setup.py | 9 ++++----- - 12 files changed, 97 insertions(+), 23 deletions(-) - -Index: Python-3.5.4/Include/pythonrun.h -=================================================================== ---- Python-3.5.4.orig/Include/pythonrun.h -+++ Python-3.5.4/Include/pythonrun.h -@@ -23,6 +23,9 @@ typedef struct { - } PyCompilerFlags; - #endif - -+PyAPI_FUNC(const char *) Py_GetArch(void); -+PyAPI_FUNC(const char *) Py_GetLib(void); -+ - #ifndef Py_LIMITED_API - PyAPI_FUNC(int) PyRun_SimpleStringFlags(const char *, PyCompilerFlags *); - PyAPI_FUNC(int) PyRun_AnyFileFlags(FILE *, const char *, PyCompilerFlags *); -Index: Python-3.5.4/Lib/distutils/command/install.py -=================================================================== ---- Python-3.5.4.orig/Lib/distutils/command/install.py -+++ Python-3.5.4/Lib/distutils/command/install.py -@@ -19,6 +19,8 @@ from site import USER_BASE - from site import USER_SITE - HAS_USER_SITE = True - -+libname = sys.lib -+ - WINDOWS_SCHEME = { - 'purelib': '$base/Lib/site-packages', - 'platlib': '$base/Lib/site-packages', -@@ -29,8 +31,8 @@ WINDOWS_SCHEME = { - - INSTALL_SCHEMES = { - 'unix_prefix': { -- 'purelib': '$base/lib/python$py_version_short/site-packages', -- 'platlib': '$platbase/lib/python$py_version_short/site-packages', -+ 'purelib': '$platbase/'+libname+'/python$py_version_short/site-packages', -+ 'platlib': '$platbase/'+libname+'/python$py_version_short/site-packages', - 'headers': '$base/include/python$py_version_short$abiflags/$dist_name', - 'scripts': '$base/bin', - 'data' : '$base', -Index: Python-3.5.4/Lib/pydoc.py -=================================================================== ---- Python-3.5.4.orig/Lib/pydoc.py -+++ Python-3.5.4/Lib/pydoc.py -@@ -389,7 +389,7 @@ class Doc: - docmodule = docclass = docroutine = docother = docproperty = docdata = fail - - def getdocloc(self, object, -- basedir=os.path.join(sys.base_exec_prefix, "lib", -+ basedir=os.path.join(sys.base_exec_prefix, sys.lib, - "python%d.%d" % sys.version_info[:2])): - """Return the location of module docs or None""" - -Index: Python-3.5.4/Lib/site.py -=================================================================== ---- Python-3.5.4.orig/Lib/site.py -+++ Python-3.5.4/Lib/site.py -@@ -303,12 +303,12 @@ def getsitepackages(prefixes=None): - seen.add(prefix) - - if os.sep == '/': -- sitepackages.append(os.path.join(prefix, "lib", -+ sitepackages.append(os.path.join(prefix, sys.lib, - "python" + sys.version[:3], - "site-packages")) - else: - sitepackages.append(prefix) -- sitepackages.append(os.path.join(prefix, "lib", "site-packages")) -+ sitepackages.append(os.path.join(prefix, sys.lib, "site-packages")) - if sys.platform == "darwin": - # for framework builds *only* we add the standard Apple - # locations. -Index: Python-3.5.4/Lib/sysconfig.py -=================================================================== ---- Python-3.5.4.orig/Lib/sysconfig.py -+++ Python-3.5.4/Lib/sysconfig.py -@@ -20,10 +20,10 @@ __all__ = [ - - _INSTALL_SCHEMES = { - 'posix_prefix': { -- 'stdlib': '{installed_base}/lib/python{py_version_short}', -- 'platstdlib': '{platbase}/lib/python{py_version_short}', -- 'purelib': '{base}/lib/python{py_version_short}/site-packages', -- 'platlib': '{platbase}/lib/python{py_version_short}/site-packages', -+ 'stdlib': '{installed_base}/'+sys.lib+'/python{py_version_short}', -+ 'platstdlib': '{platbase}/'+sys.lib+'/python{py_version_short}', -+ 'purelib': '{platbase}/'+sys.lib+'/python{py_version_short}/site-packages', -+ 'platlib': '{platbase}/'+sys.lib+'/python{py_version_short}/site-packages', - 'include': - '{installed_base}/include/python{py_version_short}{abiflags}', - 'platinclude': -@@ -32,10 +32,10 @@ _INSTALL_SCHEMES = { - 'data': '{base}', - }, - 'posix_home': { -- 'stdlib': '{installed_base}/lib/python', -- 'platstdlib': '{base}/lib/python', -- 'purelib': '{base}/lib/python', -- 'platlib': '{base}/lib/python', -+ 'stdlib': '{installed_base}/'+sys.lib+'/python', -+ 'platstdlib': '{base}/'+sys.lib+'/python', -+ 'purelib': '{base}/'+sys.lib+'/python', -+ 'platlib': '{base}/'+sys.lib+'/python', - 'include': '{installed_base}/include/python', - 'platinclude': '{installed_base}/include/python', - 'scripts': '{base}/bin', -@@ -61,10 +61,10 @@ _INSTALL_SCHEMES = { - 'data': '{userbase}', - }, - 'posix_user': { -- 'stdlib': '{userbase}/lib/python{py_version_short}', -- 'platstdlib': '{userbase}/lib/python{py_version_short}', -- 'purelib': '{userbase}/lib/python{py_version_short}/site-packages', -- 'platlib': '{userbase}/lib/python{py_version_short}/site-packages', -+ 'stdlib': '{userbase}/'+sys.lib+'/python{py_version_short}', -+ 'platstdlib': '{userbase}/'+sys.lib+'/python{py_version_short}', -+ 'purelib': '{userbase}/'+sys.lib+'/python{py_version_short}/site-packages', -+ 'platlib': '{userbase}/'+sys.lib+'/python{py_version_short}/site-packages', - 'include': '{userbase}/include/python{py_version_short}', - 'scripts': '{userbase}/bin', - 'data': '{userbase}', -Index: Python-3.5.4/Lib/trace.py -=================================================================== ---- Python-3.5.4.orig/Lib/trace.py -+++ Python-3.5.4/Lib/trace.py -@@ -749,10 +749,10 @@ def main(argv=None): - # should I also call expanduser? (after all, could use $HOME) - - s = s.replace("$prefix", -- os.path.join(sys.base_prefix, "lib", -+ os.path.join(sys.base_prefix, sys.lib, - "python" + sys.version[:3])) - s = s.replace("$exec_prefix", -- os.path.join(sys.base_exec_prefix, "lib", -+ os.path.join(sys.base_exec_prefix, sys.lib, - "python" + sys.version[:3])) - s = os.path.normpath(s) - ignore_dirs.append(s) -Index: Python-3.5.4/Makefile.pre.in -=================================================================== ---- Python-3.5.4.orig/Makefile.pre.in -+++ Python-3.5.4/Makefile.pre.in -@@ -109,6 +109,8 @@ CFLAGS_ALIASING=@CFLAGS_ALIASING@ - - # Machine-dependent subdirectories - MACHDEP= @MACHDEP@ -+LIB= @LIB@ -+ARCH= @ARCH@ - - # Multiarch directory (may be empty) - MULTIARCH= @MULTIARCH@ -@@ -128,7 +130,7 @@ LIBDIR= @libdir@ - MANDIR= @mandir@ - INCLUDEDIR= @includedir@ - CONFINCLUDEDIR= $(exec_prefix)/include --SCRIPTDIR= $(prefix)/lib -+SCRIPTDIR= @libdir@ - ABIFLAGS= @ABIFLAGS@ - - # Detailed destination directories -@@ -731,6 +733,7 @@ Modules/getpath.o: $(srcdir)/Modules/get - -DEXEC_PREFIX='"$(exec_prefix)"' \ - -DVERSION='"$(VERSION)"' \ - -DVPATH='"$(VPATH)"' \ -+ -DARCH='"$(ARCH)"' -DLIB='"$(LIB)"' \ - -o $@ $(srcdir)/Modules/getpath.c - - Programs/python.o: $(srcdir)/Programs/python.c -@@ -813,7 +816,7 @@ regen-opcode: - Python/compile.o Python/symtable.o Python/ast.o: $(srcdir)/Include/graminit.h $(srcdir)/Include/Python-ast.h - - Python/getplatform.o: $(srcdir)/Python/getplatform.c -- $(CC) -c $(PY_CORE_CFLAGS) -DPLATFORM='"$(MACHDEP)"' -o $@ $(srcdir)/Python/getplatform.c -+ $(CC) -c $(PY_CORE_CFLAGS) -DPLATFORM='"$(MACHDEP)"' -DARCH='"$(ARCH)"' -DLIB='"$(LIB)"' -o $@ $(srcdir)/Python/getplatform.c - - Python/importdl.o: $(srcdir)/Python/importdl.c - $(CC) -c $(PY_CORE_CFLAGS) -I$(DLINCLDIR) -o $@ $(srcdir)/Python/importdl.c -Index: Python-3.5.4/Modules/getpath.c -=================================================================== ---- Python-3.5.4.orig/Modules/getpath.c -+++ Python-3.5.4/Modules/getpath.c -@@ -105,6 +105,13 @@ - #error "PREFIX, EXEC_PREFIX, VERSION, and VPATH must be constant defined" - #endif - -+#define LIB_PYTHON LIB "/python" VERSION -+ -+#ifndef PYTHONPATH -+#define PYTHONPATH PREFIX "/" LIB_PYTHON ":" \ -+ EXEC_PREFIX "/" LIB_PYTHON "/lib-dynload" -+#endif -+ - #ifndef LANDMARK - #define LANDMARK L"os.py" - #endif -@@ -113,6 +120,7 @@ static wchar_t prefix[MAXPATHLEN+1]; - static wchar_t exec_prefix[MAXPATHLEN+1]; - static wchar_t progpath[MAXPATHLEN+1]; - static wchar_t *module_search_path = NULL; -+static wchar_t *lib_python = L"" LIB_PYTHON; - - /* Get file status. Encode the path to the locale encoding. */ - -@@ -494,7 +502,7 @@ calculate_path(void) - _pythonpath = Py_DecodeLocale(PYTHONPATH, NULL); - _prefix = Py_DecodeLocale(PREFIX, NULL); - _exec_prefix = Py_DecodeLocale(EXEC_PREFIX, NULL); -- lib_python = Py_DecodeLocale("lib/python" VERSION, NULL); -+ lib_python = Py_DecodeLocale(LIB_PYTHON, NULL); - - if (!_pythonpath || !_prefix || !_exec_prefix || !lib_python) { - Py_FatalError( -Index: Python-3.5.4/Python/getplatform.c -=================================================================== ---- Python-3.5.4.orig/Python/getplatform.c -+++ Python-3.5.4/Python/getplatform.c -@@ -10,3 +10,23 @@ Py_GetPlatform(void) - { - return PLATFORM; - } -+ -+#ifndef ARCH -+#define ARCH "unknown" -+#endif -+ -+const char * -+Py_GetArch(void) -+{ -+ return ARCH; -+} -+ -+#ifndef LIB -+#define LIB "lib" -+#endif -+ -+const char * -+Py_GetLib(void) -+{ -+ return LIB; -+} -Index: Python-3.5.4/Python/sysmodule.c -=================================================================== ---- Python-3.5.4.orig/Python/sysmodule.c -+++ Python-3.5.4/Python/sysmodule.c -@@ -1827,6 +1827,10 @@ _PySys_Init(void) - PyUnicode_FromString(Py_GetCopyright())); - SET_SYS_FROM_STRING("platform", - PyUnicode_FromString(Py_GetPlatform())); -+ SET_SYS_FROM_STRING("arch", -+ PyUnicode_FromString(Py_GetArch())); -+ SET_SYS_FROM_STRING("lib", -+ PyUnicode_FromString(Py_GetLib())); - SET_SYS_FROM_STRING("executable", - PyUnicode_FromWideChar( - Py_GetProgramFullPath(), -1)); -Index: Python-3.5.4/configure.ac -=================================================================== ---- Python-3.5.4.orig/configure.ac -+++ Python-3.5.4/configure.ac -@@ -885,6 +885,41 @@ PLATDIR=plat-$MACHDEP - AC_SUBST(PLATDIR) - AC_SUBST(PLATFORM_TRIPLET) - -+AC_SUBST(ARCH) -+AC_MSG_CHECKING(ARCH) -+ARCH=`uname -m` -+case $ARCH in -+i?86) ARCH=i386;; -+esac -+AC_MSG_RESULT($ARCH) -+ -+AC_SUBST(LIB) -+AC_MSG_CHECKING(LIB) -+case $ac_sys_system in -+Linux*) -+ # Test if the compiler is 64bit -+ echo 'int i;' > conftest.$ac_ext -+ python_cv_cc_64bit_output=no -+ if AC_TRY_EVAL(ac_compile); then -+ case `/usr/bin/file conftest.$ac_objext` in -+ *"ELF 64"*) -+ python_cv_cc_64bit_output=yes -+ ;; -+ esac -+ fi -+ rm -rf conftest* -+ ;; -+esac -+ -+case $ARCH:$python_cv_cc_64bit_output in -+ppc64:yes | powerpc64:yes | s390x:yes | sparc64:yes | x86_64:yes) -+ LIB="lib64" -+ ;; -+*:*) -+ LIB="lib" -+ ;; -+esac -+AC_MSG_RESULT($LIB) - - AC_MSG_CHECKING([for -Wl,--no-as-needed]) - save_LDFLAGS="$LDFLAGS" -Index: Python-3.5.4/setup.py -=================================================================== ---- Python-3.5.4.orig/setup.py -+++ Python-3.5.4/setup.py -@@ -494,7 +494,7 @@ class PyBuildExt(build_ext): - # directories (i.e. '.' and 'Include') must be first. See issue - # 10520. - if not cross_compiling: -- add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib') -+ add_dir_to_list(self.compiler.library_dirs, os.path.join('/usr/local', sys.lib)) - add_dir_to_list(self.compiler.include_dirs, '/usr/local/include') - # only change this for cross builds for 3.3, issues on Mageia - if cross_compiling: -@@ -552,8 +552,7 @@ class PyBuildExt(build_ext): - # be assumed that no additional -I,-L directives are needed. - if not cross_compiling: - lib_dirs = self.compiler.library_dirs + [ -- '/lib64', '/usr/lib64', -- '/lib', '/usr/lib', -+ '/' + sys.lib, '/usr/' + sys.lib, - ] - inc_dirs = self.compiler.include_dirs + ['/usr/include'] - else: -@@ -745,11 +744,11 @@ class PyBuildExt(build_ext): - elif curses_library: - readline_libs.append(curses_library) - elif self.compiler.find_library_file(lib_dirs + -- ['/usr/lib/termcap'], -+ ['/usr/'+sys.lib+'/termcap'], - 'termcap'): - readline_libs.append('termcap') - exts.append( Extension('readline', ['readline.c'], -- library_dirs=['/usr/lib/termcap'], -+ library_dirs=['/usr/'+sys.lib+'/termcap'], - extra_link_args=readline_extra_link_args, - libraries=readline_libs) ) - else: diff --git a/meta/recipes-devtools/python/python3/python-config.patch b/meta/recipes-devtools/python/python3/python-config.patch deleted file mode 100644 index f23b8b7df06..00000000000 --- a/meta/recipes-devtools/python/python3/python-config.patch +++ /dev/null @@ -1,46 +0,0 @@ -python-config: Revert to using distutils.sysconfig - -The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in - -12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig - -affect the native runtime as well as cross building. Use the old, patched -implementation which returns paths in the staging directory and for the target, -as appropriate. - -Upstream-Status: Inappropriate [Embedded Specific] - -Signed-off-by: Tyler Hall <tylerwhall@gmail.com> -: -Index: Python-3.3.3/Misc/python-config.in -=================================================================== ---- Python-3.3.3.orig/Misc/python-config.in -+++ Python-3.3.3/Misc/python-config.in -@@ -4,7 +4,7 @@ - import getopt - import os - import sys --import sysconfig -+from distutils import sysconfig - - valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', - 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] -@@ -32,14 +32,14 @@ if '--help' in opt_flags: - - for opt in opt_flags: - if opt == '--prefix': -- print(sysconfig.get_config_var('prefix')) -+ print(sysconfig.PREFIX) - - elif opt == '--exec-prefix': -- print(sysconfig.get_config_var('exec_prefix')) -+ print(sysconfig.EXEC_PREFIX) - - elif opt in ('--includes', '--cflags'): -- flags = ['-I' + sysconfig.get_path('include'), -- '-I' + sysconfig.get_path('platinclude')] -+ flags = ['-I' + sysconfig.get_python_inc(), -+ '-I' + sysconfig.get_python_inc(plat_specific=True)] - if opt == '--cflags': - flags.extend(getvar('CFLAGS').split()) - print(' '.join(flags)) diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json deleted file mode 100644 index 3641d5bd7b1..00000000000 --- a/meta/recipes-devtools/python/python3/python3-manifest.json +++ /dev/null @@ -1,1242 +0,0 @@ -# DO NOT (entirely) modify this file manually, please read. -# -# IMPORTANT NOTE: -# Please keep in mind that the create_manifest task relies on the fact the the -# target and native Python packages are the same, and it also needs to be executed -# with a fully working native package (with all the PACKAGECONFIGs enabled and all -# and all the modules should be working, check log.do_compile), otherwise the script -# will fail to find dependencies correctly, this note is valid either if you are -# upgrading to a new Python version or adding a new package. -# -# -# If you are adding a new package please follow the next steps: -# How to add a new package: -# - If a user wants to add a new package all that has to be done is: -# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, -# fill up the SUMMARY section as well, the script should handle all the rest. -# -# Real example: -# We want to add a web browser package, including the file webbrowser.py -# which at the moment is on python3-misc. -# "webbrowser": { -# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], -# "rdepends": [], -# "summary": "Python Web Browser support"} -# -# * Note that the rdepends field was left empty -# -# We run $ bitbake python3 -c create_manifest and the resulting manifest -# should be completed after a few seconds, showing something like: -# "webbrowser": { -# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], -# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], -# "summary": "Python Web Browser support"} -# -# -# If you are upgrading Python to a new version please follow the next steps: -# After each Python upgrade, the create_manifest task should be executed, because we -# don't control what changes on upstream Python, so, some module dependency -# might have changed without us realizing it, a certain module can either have -# more or less dependencies, or could be depending on a new file that was just -# created on the new release and for obvious reasons we wouldn't have it on our -# old manifest, all of these issues would cause runtime errors on our system. -# -# - Upgrade both the native and target Python packages to a new version -# - Run the create_manifest task for the target Python package as its shown below: -# -# $ bitbake python3 -c create_manifest -# -# This will automatically replace your manifest file located under the Python directory -# with an new one, which contains the new dependencies (if any). -# -# Several things could have gone wrong here, I will try to explain a few: -# -# a) A new file was introduced on this release, e.g. sha3*.so: -# The task will check what its needed to import every module, more than one module would -# would probably depend on sha3*.so, although only one module should contain it. -# -# After running the task, the new manifest will have the sha3*.so file on more than one -# module, you need to manually decide which one of them should get it and delete it from -# the others, for example sha3*.so should likely be on ${PN}-crypt. -# Once you have deleted from the others you need to run the create_manifest task again, -# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be -# good to go. -# -# b) The native package wasn't built correctly and its missing a certain module: -# As mentioned before, you need to make sure the native package was built with all the modules -# because it is used as base to build the manifest file, you need to manually check log.do_compile -# since it won't error out the compile function if its only missing a couple of modules. -# -# e.g. missing the _uuid module, log.do_compile would show the following: -# Python build finished successfully! -# The necessary bits to build these optional modules were not found: -# _uuid -# -# What will happen here is that the new manifest would not be aware that the _uuid module exists, so -# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on -# the misc package (which is where any file that doesn't belong anywhere else ends up). -# -# This will eventually cause runtime errors on our system if we don't include the misc package on -# on our image, because the _uuid files will be missing. -# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be -# detected correctly along with its dependencies, and we will get a working manifest. -# -# This is the reason why it is important to make sure we have a fully working native build, -# so we can avoid these errors. -# -# -# -# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST -# EOC -{ - "tests": { - "summary": "Python test suite", - "rdepends": [ - "core", - "modules" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/*/test", - "${libdir}/python${PYTHON_MAJMIN}/*/tests", - "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", - "${libdir}/python${PYTHON_MAJMIN}/test" - ], - "cached": [] - }, - "2to3": { - "summary": "Python automated Python 2 to 3 code translator", - "rdepends": [ - "core" - ], - "files": [ - "${bindir}/2to3-*", - "${libdir}/python${PYTHON_MAJMIN}/lib2to3" - ], - "cached": [] - }, - "asyncio": { - "summary": "Python Asynchronous I/", - "rdepends": [ - "compression", - "core", - "crypt", - "io", - "logging", - "math", - "multiprocessing", - "netclient", - "pickle", - "shell", - "stringold", - "threading", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/asyncio", - "${libdir}/python${PYTHON_MAJMIN}/concurrent", - "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures" - ], - "cached": [] - }, - "audio": { - "summary": "Python Audio Handling", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/chunk.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", - "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", - "${libdir}/python${PYTHON_MAJMIN}/sunau.py", - "${libdir}/python${PYTHON_MAJMIN}/wave.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" - ] - }, - "codecs": { - "summary": "Python codec", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", - "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" - ] - }, - "compile": { - "summary": "Python bytecode compilation support", - "rdepends": [ - "asyncio", - "compression", - "core", - "crypt", - "io", - "logging", - "math", - "multiprocessing", - "pickle", - "shell", - "stringold", - "threading", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/compileall.py", - "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" - ] - }, - "compression": { - "summary": "Python high-level compression support", - "rdepends": [ - "core", - "shell", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_compression.py", - "${libdir}/python${PYTHON_MAJMIN}/bz2.py", - "${libdir}/python${PYTHON_MAJMIN}/gzip.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lzma.py", - "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", - "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" - ] - }, - "core": { - "summary": "Python interpreter and core modules", - "rdepends": [], - "files": [ - "${bindir}/python*[!-config]", - "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", - "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", - "${libdir}/python${PYTHON_MAJMIN}/UserList.py", - "${libdir}/python${PYTHON_MAJMIN}/UserString.py", - "${libdir}/python${PYTHON_MAJMIN}/__future__.py", - "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", - "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", - "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", - "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", - "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", - "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", - "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", - "${libdir}/python${PYTHON_MAJMIN}/abc.py", - "${libdir}/python${PYTHON_MAJMIN}/argparse.py", - "${libdir}/python${PYTHON_MAJMIN}/ast.py", - "${libdir}/python${PYTHON_MAJMIN}/bisect.py", - "${libdir}/python${PYTHON_MAJMIN}/code.py", - "${libdir}/python${PYTHON_MAJMIN}/codecs.py", - "${libdir}/python${PYTHON_MAJMIN}/codeop.py", - "${libdir}/python${PYTHON_MAJMIN}/collections", - "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", - "${libdir}/python${PYTHON_MAJMIN}/configparser.py", - "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", - "${libdir}/python${PYTHON_MAJMIN}/copy.py", - "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", - "${libdir}/python${PYTHON_MAJMIN}/csv.py", - "${libdir}/python${PYTHON_MAJMIN}/dis.py", - "${libdir}/python${PYTHON_MAJMIN}/encodings", - "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", - "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", - "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", - "${libdir}/python${PYTHON_MAJMIN}/enum.py", - "${libdir}/python${PYTHON_MAJMIN}/functools.py", - "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", - "${libdir}/python${PYTHON_MAJMIN}/getopt.py", - "${libdir}/python${PYTHON_MAJMIN}/gettext.py", - "${libdir}/python${PYTHON_MAJMIN}/heapq.py", - "${libdir}/python${PYTHON_MAJMIN}/imp.py", - "${libdir}/python${PYTHON_MAJMIN}/importlib", - "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", - "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", - "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", - "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", - "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", - "${libdir}/python${PYTHON_MAJMIN}/inspect.py", - "${libdir}/python${PYTHON_MAJMIN}/io.py", - "${libdir}/python${PYTHON_MAJMIN}/keyword.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", - "${libdir}/python${PYTHON_MAJMIN}/linecache.py", - "${libdir}/python${PYTHON_MAJMIN}/locale.py", - "${libdir}/python${PYTHON_MAJMIN}/new.py", - "${libdir}/python${PYTHON_MAJMIN}/opcode.py", - "${libdir}/python${PYTHON_MAJMIN}/operator.py", - "${libdir}/python${PYTHON_MAJMIN}/optparse.py", - "${libdir}/python${PYTHON_MAJMIN}/os.py", - "${libdir}/python${PYTHON_MAJMIN}/platform.py", - "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", - "${libdir}/python${PYTHON_MAJMIN}/re.py", - "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", - "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", - "${libdir}/python${PYTHON_MAJMIN}/selectors.py", - "${libdir}/python${PYTHON_MAJMIN}/signal.py", - "${libdir}/python${PYTHON_MAJMIN}/site.py", - "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", - "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", - "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", - "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", - "${libdir}/python${PYTHON_MAJMIN}/stat.py", - "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", - "${libdir}/python${PYTHON_MAJMIN}/struct.py", - "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", - "${libdir}/python${PYTHON_MAJMIN}/symbol.py", - "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", - "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", - "${libdir}/python${PYTHON_MAJMIN}/threading.py", - "${libdir}/python${PYTHON_MAJMIN}/token.py", - "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", - "${libdir}/python${PYTHON_MAJMIN}/traceback.py", - "${libdir}/python${PYTHON_MAJMIN}/types.py", - "${libdir}/python${PYTHON_MAJMIN}/warnings.py", - "${libdir}/python${PYTHON_MAJMIN}/weakref.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", - "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", - "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", - "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" - ] - }, - "crypt": { - "summary": "Python basic cryptographic and hashing support", - "rdepends": [ - "core", - "math", - "stringold" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/crypt.py", - "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" - ] - }, - "ctypes": { - "summary": "Python C types support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/ctypes", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" - ], - "cached": [] - }, - "curses": { - "summary": "Python curses support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/curses", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" - ], - "cached": [] - }, - "datetime": { - "summary": "Python calendar and time support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", - "${libdir}/python${PYTHON_MAJMIN}/calendar.py", - "${libdir}/python${PYTHON_MAJMIN}/datetime.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" - ] - }, - "db": { - "summary": "Python file-based database support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/dbm", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" - ], - "cached": [] - }, - "debugger": { - "summary": "Python debugger", - "rdepends": [ - "core", - "pprint", - "shell", - "stringold" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/bdb.py", - "${libdir}/python${PYTHON_MAJMIN}/pdb.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" - ] - }, - "dev": { - "cached": [], - "files": [ - "${base_libdir}/*.a", - "${base_libdir}/*.o", - "${bindir}/python*-config", - "${datadir}/aclocal", - "${datadir}/pkgconfig", - "${includedir}", - "${libdir}/*.a", - "${libdir}/*.la", - "${libdir}/*.o", - "${libdir}/lib*${SOLIBSDEV}", - "${libdir}/pkgconfig", - "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", - "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" - ], - "rdepends": [ - "core" - ], - "summary": "Python development package" - }, - "difflib": { - "summary": "Python helpers for computing deltas between objects", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/difflib.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" - ] - }, - "distutils-staticdev": { - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" - ], - "rdepends": [ - "distutils" - ], - "summary": "Python distribution utilities (static libraries)" - }, - "distutils": { - "summary": "Python Distribution Utilities", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/distutils" - ], - "cached": [] - }, - "doctest": { - "summary": "Python framework for running examples in docstrings", - "rdepends": [ - "core", - "debugger", - "difflib", - "logging", - "pprint", - "shell", - "stringold", - "unittest" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/doctest.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" - ] - }, - "email": { - "summary": "Python email support", - "rdepends": [ - "core", - "crypt", - "datetime", - "io", - "math", - "netclient" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/email", - "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" - ] - }, - "fcntl": { - "summary": "Python's fcntl interface", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" - ], - "cached": [] - }, - "gdbm": { - "summary": "Python GNU database support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" - ], - "cached": [] - }, - "html": { - "summary": "Python HTML processing support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/formatter.py", - "${libdir}/python${PYTHON_MAJMIN}/html" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" - ] - }, - "idle": { - "summary": "Python Integrated Development Environment", - "rdepends": [ - "core" - ], - "files": [ - "${bindir}/idle*", - "${libdir}/python${PYTHON_MAJMIN}/idlelib" - ], - "cached": [] - }, - "image": { - "summary": "Python graphical image handling", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", - "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" - ] - }, - "io": { - "summary": "Python low-level I/O", - "rdepends": [ - "compression", - "core", - "crypt", - "math", - "netclient", - "shell", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", - "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", - "${libdir}/python${PYTHON_MAJMIN}/pipes.py", - "${libdir}/python${PYTHON_MAJMIN}/socket.py", - "${libdir}/python${PYTHON_MAJMIN}/ssl.py", - "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" - ] - }, - "json": { - "summary": "Python JSON support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/json", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" - ], - "cached": [] - }, - "logging": { - "summary": "Python logging support", - "rdepends": [ - "core", - "stringold" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/logging" - ], - "cached": [] - }, - "mailbox": { - "summary": "Python mailbox format support", - "rdepends": [ - "core", - "crypt", - "datetime", - "email", - "fcntl", - "io", - "math", - "mime", - "netclient", - "stringold" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" - ] - }, - "math": { - "summary": "Python math support", - "rdepends": [ - "core", - "crypt" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", - "${libdir}/python${PYTHON_MAJMIN}/random.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" - ] - }, - "mime": { - "summary": "Python MIME handling APIs", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/quopri.py", - "${libdir}/python${PYTHON_MAJMIN}/uu.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" - ] - }, - "mmap": { - "summary": "Python memory-mapped file support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" - ], - "cached": [] - }, - "modules": { - "cached": [], - "files": [], - "rdepends": [ - "2to3", - "asyncio", - "audio", - "codecs", - "compile", - "compression", - "core", - "crypt", - "ctypes", - "curses", - "datetime", - "db", - "debugger", - "difflib", - "distutils", - "doctest", - "email", - "fcntl", - "html", - "idle", - "image", - "io", - "json", - "logging", - "mailbox", - "math", - "mime", - "mmap", - "multiprocessing", - "netclient", - "netserver", - "numbers", - "pickle", - "pkgutil", - "plistlib", - "pprint", - "profile", - "pydoc", - "resource", - "runpy", - "shell", - "smtpd", - "sqlite3", - "stringold", - "syslog", - "terminal", - "threading", - "tkinter", - "typing", - "unittest", - "unixadmin", - "venv", - "xml", - "xmlrpc" - ], - "summary": "All Python modules" - }, - "multiprocessing": { - "summary": "Python multiprocessing support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", - "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" - ], - "cached": [] - }, - "netclient": { - "summary": "Python Internet Protocol clients", - "rdepends": [ - "compression", - "core", - "crypt", - "ctypes", - "datetime", - "email", - "io", - "math", - "mime", - "shell", - "stringold", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/base64.py", - "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", - "${libdir}/python${PYTHON_MAJMIN}/hmac.py", - "${libdir}/python${PYTHON_MAJMIN}/http", - "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", - "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", - "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", - "${libdir}/python${PYTHON_MAJMIN}/poplib.py", - "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", - "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", - "${libdir}/python${PYTHON_MAJMIN}/urllib", - "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", - "${libdir}/python${PYTHON_MAJMIN}/uuid.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" - ] - }, - "netserver": { - "summary": "Python Internet Protocol servers", - "rdepends": [ - "compression", - "core", - "crypt", - "datetime", - "email", - "html", - "io", - "math", - "mime", - "netclient", - "shell", - "stringold", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/cgi.py", - "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" - ] - }, - "numbers": { - "summary": "Python number APIs", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", - "${libdir}/python${PYTHON_MAJMIN}/decimal.py", - "${libdir}/python${PYTHON_MAJMIN}/fractions.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", - "${libdir}/python${PYTHON_MAJMIN}/numbers.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" - ] - }, - "pickle": { - "summary": "Python serialisation/persistence support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", - "${libdir}/python${PYTHON_MAJMIN}/pickle.py", - "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", - "${libdir}/python${PYTHON_MAJMIN}/shelve.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" - ] - }, - "pkgutil": { - "summary": "Python package extension utility support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" - ] - }, - "plistlib": { - "summary": "Generate and parse Mac OS X .plist files", - "rdepends": [ - "core", - "datetime", - "xml" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" - ] - }, - "pprint": { - "summary": "Python pretty-print support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/pprint.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" - ] - }, - "profile": { - "summary": "Python basic performance profiling support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", - "${libdir}/python${PYTHON_MAJMIN}/profile.py", - "${libdir}/python${PYTHON_MAJMIN}/pstats.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" - ] - }, - "pydoc": { - "summary": "Python interactive help support", - "rdepends": [ - "core", - "netclient", - "pkgutil" - ], - "files": [ - "${bindir}/pydoc*", - "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", - "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" - ] - }, - "resource": { - "summary": "Python resource control interface", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" - ], - "cached": [] - }, - "runpy": { - "summary": "Python helper for locating/executing scripts in module namespace", - "rdepends": [ - "core", - "pkgutil" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/runpy.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" - ] - }, - "shell": { - "summary": "Python shell-like functionality", - "rdepends": [ - "compression", - "core", - "stringold", - "unixadmin" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/cmd.py", - "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", - "${libdir}/python${PYTHON_MAJMIN}/glob.py", - "${libdir}/python${PYTHON_MAJMIN}/shlex.py", - "${libdir}/python${PYTHON_MAJMIN}/shutil.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" - ] - }, - "smtpd": { - "summary": "Python Simple Mail Transport Daemon", - "rdepends": [ - "core", - "crypt", - "datetime", - "email", - "io", - "math", - "mime", - "netclient", - "stringold" - ], - "files": [ - "${bindir}/smtpd.py", - "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", - "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", - "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" - ] - }, - "sqlite3": { - "summary": "Python Sqlite3 database support", - "rdepends": [ - "core", - "datetime" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", - "${libdir}/python${PYTHON_MAJMIN}/sqlite3" - ], - "cached": [] - }, - "stringold": { - "summary": "Python string APIs [deprecated]", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/string.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" - ] - }, - "syslog": { - "summary": "Python syslog interface", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" - ], - "cached": [] - }, - "terminal": { - "summary": "Python terminal controlling support", - "rdepends": [ - "core", - "io" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/pty.py", - "${libdir}/python${PYTHON_MAJMIN}/tty.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" - ] - }, - "threading": { - "summary": "Python threading & synchronization support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", - "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", - "${libdir}/python${PYTHON_MAJMIN}/dummy_threading.py", - "${libdir}/python${PYTHON_MAJMIN}/queue.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dummy_threading.*.pyc", - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" - ] - }, - "tkinter": { - "summary": "Python Tcl/Tk bindings", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", - "${libdir}/python${PYTHON_MAJMIN}/tkinter" - ], - "cached": [] - }, - "typing": { - "summary": "Python typing support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/typing.py" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" - ] - }, - "unittest": { - "summary": "Python unit testing framework", - "rdepends": [ - "core", - "difflib", - "logging", - "pprint", - "shell", - "stringold" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/unittest", - "${libdir}/python${PYTHON_MAJMIN}/unittest/", - "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" - ], - "cached": [] - }, - "unixadmin": { - "summary": "Python Unix administration support", - "rdepends": [ - "core", - "io" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/getpass.py", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" - ], - "cached": [ - "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" - ] - }, - "venv": { - "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", - "rdepends": [ - "compression", - "core", - "logging", - "shell", - "stringold", - "unixadmin" - ], - "files": [ - "${bindir}/pyvenv*", - "${libdir}/python${PYTHON_MAJMIN}/venv" - ], - "cached": [] - }, - "xml": { - "summary": "Python basic XML support", - "rdepends": [ - "core" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", - "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", - "${libdir}/python${PYTHON_MAJMIN}/xml" - ], - "cached": [] - }, - "xmlrpc": { - "summary": "Python XML-RPC support", - "rdepends": [ - "core", - "xml" - ], - "files": [ - "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", - "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" - ], - "cached": [] - } -} diff --git a/meta/recipes-devtools/python/python3/python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch b/meta/recipes-devtools/python/python3/python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch deleted file mode 100644 index a4f8bd47107..00000000000 --- a/meta/recipes-devtools/python/python3/python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch +++ /dev/null @@ -1,25 +0,0 @@ -From 53ed216d7bf70dd2a925432b6805a701e5fc3e0e Mon Sep 17 00:00:00 2001 -From: Jackie Huang <jackie.huang@windriver.com> -Date: Mon, 17 Nov 2014 06:44:47 +0000 -Subject: [PATCH] python3 use CROSSPYTHONPATH for PYTHON_FOR_BUILD - -Upstream-Status: Inappropriate [Cross compile specific] - -Signed-off-by: Jackie Huang <jackie.huang@windriver.com> ---- - configure.ac | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -Index: Python-3.5.4/configure.ac -=================================================================== ---- Python-3.5.4.orig/configure.ac -+++ Python-3.5.4/configure.ac -@@ -73,7 +73,7 @@ if test "$cross_compiling" = yes; then - AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found]) - fi - AC_MSG_RESULT($interp) -- PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp -+ PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH) '$interp - fi - elif test "$cross_compiling" = maybe; then - AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH]) diff --git a/meta/recipes-devtools/python/python3/regen-all.patch b/meta/recipes-devtools/python/python3/regen-all.patch deleted file mode 100644 index 36b9d9d9196..00000000000 --- a/meta/recipes-devtools/python/python3/regen-all.patch +++ /dev/null @@ -1,25 +0,0 @@ -python3-native: run regen-importlib target correctly - -regen-importlib depends on other regen- targets, so we must be sure to -run it after (most of) the others. In reality, we do not need to run it -at all since "make" will invoke it, if necessary. We do not want to -rely on that, though. - -Upstream-Status: Pending - -Signed-off-by: Joe Slater <joe.slater@windriver.com> - - - ---- a/Makefile.pre.in -+++ b/Makefile.pre.in -@@ -709,7 +709,8 @@ regen-importlib: Programs/_freeze_import - ############################################################################ - # Regenerate all generated files - --regen-all: regen-opcode regen-opcode-targets regen-typeslots regen-grammar regen-ast regen-importlib -+regen-all: regen-opcode regen-opcode-targets regen-typeslots regen-grammar regen-ast -+ $(MAKE) regen-importlib - - ############################################################################ - # Special rules for object files diff --git a/meta/recipes-devtools/python/python3/run-ptest b/meta/recipes-devtools/python/python3/run-ptest deleted file mode 100644 index 3863c6d314f..00000000000 --- a/meta/recipes-devtools/python/python3/run-ptest +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh - -python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' diff --git a/meta/recipes-devtools/python/python3/setup.py-check-cross_compiling-when-get-FLAGS.patch b/meta/recipes-devtools/python/python3/setup.py-check-cross_compiling-when-get-FLAGS.patch deleted file mode 100644 index fc2e12dfe5c..00000000000 --- a/meta/recipes-devtools/python/python3/setup.py-check-cross_compiling-when-get-FLAGS.patch +++ /dev/null @@ -1,50 +0,0 @@ -From 8dad810f3a3d073f09ad72e1a3ee0a895eab2ca1 Mon Sep 17 00:00:00 2001 -From: Robert Yang <liezhi.yang@windriver.com> -Date: Sun, 18 Jan 2015 19:05:36 -0800 -Subject: [PATCH] setup.py:check cross_compiling when get FLAGS - -Fixed when compile target pythnon3: -gcc -isystem/path/to/sysroots/x86_64-linux/usr/include \ - -L=/path/to/sysroots/x86_64-linux/usr/lib - -This is incorrect, the native sysroot should not be used by target -python3. - -Upstream-Status: Pending - -Signed-off-by: Robert Yang <liezhi.yang@windriver.com> ---- - setup.py | 10 ++++++++-- - 1 file changed, 8 insertions(+), 2 deletions(-) - -diff --git a/setup.py b/setup.py -index e8339cd..83fd31f 100644 ---- a/setup.py -+++ b/setup.py -@@ -238,7 +238,10 @@ class PyBuildExt(build_ext): - # unfortunately, distutils doesn't let us provide separate C and C++ - # compilers - if compiler is not None: -- (ccshared,cflags) = sysconfig.get_config_vars('CCSHARED','CFLAGS') -+ if cross_compiling: -+ (ccshared,cflags) = (os.environ.get('CCSHARED') or '', os.environ.get('CFLAGS') or '') -+ else: -+ (ccshared,cflags) = sysconfig.get_config_vars('CCSHARED','CFLAGS') - args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags - self.compiler.set_executables(**args) - -@@ -457,7 +460,10 @@ class PyBuildExt(build_ext): - ('LDFLAGS', '-R', self.compiler.runtime_library_dirs), - ('LDFLAGS', '-L', self.compiler.library_dirs), - ('CPPFLAGS', '-I', self.compiler.include_dirs)): -- env_val = sysconfig.get_config_var(env_var) -+ if cross_compiling: -+ env_val = os.environ.get(env_var) -+ else: -+ env_val = sysconfig.get_config_var(env_var) - if env_val: - # To prevent optparse from raising an exception about any - # options in env_val that it doesn't know about we strip out --- -1.7.9.5 - diff --git a/meta/recipes-devtools/python/python3/setuptweaks.patch b/meta/recipes-devtools/python/python3/setuptweaks.patch deleted file mode 100644 index 3a91b1916c0..00000000000 --- a/meta/recipes-devtools/python/python3/setuptweaks.patch +++ /dev/null @@ -1,57 +0,0 @@ -This patch removes various ways native system options can pass into the python -compilation and somehow break C modules. - -Upstream-Status: Inappropriate [OE Specific] - -RP 2012/04/23 - -Index: Python-2.7.2/setup.py -=================================================================== ---- Python-2.7.2.orig/setup.py 2012-04-23 20:03:47.295582553 +0000 -+++ Python-2.7.2/setup.py 2012-04-23 20:03:15.000000000 +0000 -@@ -231,7 +231,13 @@ - # compilers - if compiler is not None: - (ccshared,cflags) = sysconfig.get_config_vars('CCSHARED','CFLAGS') -- args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags -+ # Need to filter out -isysroot from the flags. Ideally should -+ # figure out target flags here. -+ flags = [] -+ for f in cflags.split(): -+ if not f.startswith("-isystem"): -+ flags.append(f) -+ args['compiler_so'] = compiler + ' ' + ccshared + ' ' + ' '.join(flags) - self.compiler.set_executables(**args) - - build_ext.build_extensions(self) -@@ -393,7 +399,6 @@ - # into configure and stored in the Makefile (issue found on OS X 10.3). - for env_var, arg_name, dir_list in ( - ('LDFLAGS', '-R', self.compiler.runtime_library_dirs), -- ('LDFLAGS', '-L', self.compiler.library_dirs), - ('CPPFLAGS', '-I', self.compiler.include_dirs)): - env_val = sysconfig.get_config_var(env_var) - if env_val: -@@ -419,16 +424,16 @@ - for directory in reversed(options.dirs): - add_dir_to_list(dir_list, directory) - -- if os.path.normpath(sys.prefix) != '/usr' \ -- and not sysconfig.get_config_var('PYTHONFRAMEWORK'): -+# if os.path.normpath(sys.prefix) != '/usr' \ -+# and not sysconfig.get_config_var('PYTHONFRAMEWORK'): - # OSX note: Don't add LIBDIR and INCLUDEDIR to building a framework - # (PYTHONFRAMEWORK is set) to avoid # linking problems when - # building a framework with different architectures than - # the one that is currently installed (issue #7473) -- add_dir_to_list(self.compiler.library_dirs, -- sysconfig.get_config_var("LIBDIR")) -- add_dir_to_list(self.compiler.include_dirs, -- sysconfig.get_config_var("INCLUDEDIR")) -+# add_dir_to_list(self.compiler.library_dirs, -+# sysconfig.get_config_var("LIBDIR")) -+# add_dir_to_list(self.compiler.include_dirs, -+# sysconfig.get_config_var("INCLUDEDIR")) - - try: - have_unicode = unicode diff --git a/meta/recipes-devtools/python/python3/signal.patch b/meta/recipes-devtools/python/python3/signal.patch deleted file mode 100644 index 534a097771c..00000000000 --- a/meta/recipes-devtools/python/python3/signal.patch +++ /dev/null @@ -1,56 +0,0 @@ -Upstream-Status: Backport -Signed-off-by: Ross Burton <ross.burton@intel.com> - -From 4315389df3c4e8c1f94a18ab11a4b234762132b1 Mon Sep 17 00:00:00 2001 -From: Antoine Pitrou <pitrou@free.fr> -Date: Mon, 23 Apr 2018 22:22:49 +0200 -Subject: [PATCH] [3.6] bpo-33329: Fix multiprocessing regression on newer - glibcs (GH-6575) (GH-6582) - -Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some -reserved signal numbers between 1 and NSIG. The `range(1, NSIG)` idiom -is commonly used to select all signals for blocking with `pthread_sigmask`. -So we ignore the sigaddset() return value until we expose sigfillset() -to provide a better idiom. -(cherry picked from commit 25038ecfb665bef641abf8cb61afff7505b0e008) ---- - .../next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst | 1 + - Modules/signalmodule.c | 14 ++++++++------ - 2 files changed, 9 insertions(+), 6 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2018-04-23-13-21-39.bpo-33329.lQ-Eod.rst - -diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c -index e0d06b434d..138e74e8a9 100644 ---- a/Modules/signalmodule.c -+++ b/Modules/signalmodule.c -@@ -744,7 +744,6 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask) - int result = -1; - PyObject *iterator, *item; - long signum; -- int err; - - sigemptyset(mask); - -@@ -766,11 +765,14 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask) - Py_DECREF(item); - if (signum == -1 && PyErr_Occurred()) - goto error; -- if (0 < signum && signum < NSIG) -- err = sigaddset(mask, (int)signum); -- else -- err = 1; -- if (err) { -+ if (0 < signum && signum < NSIG) { -+ /* bpo-33329: ignore sigaddset() return value as it can fail -+ * for some reserved signals, but we want the `range(1, NSIG)` -+ * idiom to allow selecting all valid signals. -+ */ -+ (void) sigaddset(mask, (int)signum); -+ } -+ else { - PyErr_Format(PyExc_ValueError, - "signal number %ld out of range", signum); - goto error; --- -2.11.0 - diff --git a/meta/recipes-devtools/python/python3/sitecustomize.py b/meta/recipes-devtools/python/python3/sitecustomize.py deleted file mode 100644 index 4c8b5e2ba3d..00000000000 --- a/meta/recipes-devtools/python/python3/sitecustomize.py +++ /dev/null @@ -1,37 +0,0 @@ -# OpenEmbedded sitecustomize.py (C) 2002-2008 Michael 'Mickey' Lauer <mlauer@vanille-media.de> -# GPLv2 or later -# Version: 20081123 -# Features: -# * set proper default encoding -# * enable readline completion in the interactive interpreter -# * load command line history on startup -# * save command line history on exit - -import os - -def __exithandler(): - try: - readline.write_history_file( "%s/.python-history" % os.getenv( "HOME", "/tmp" ) ) - except IOError: - pass - -def __registerExitHandler(): - import atexit - atexit.register( __exithandler ) - -def __enableReadlineSupport(): - readline.set_history_length( 1000 ) - readline.parse_and_bind( "tab: complete" ) - try: - readline.read_history_file( "%s/.python-history" % os.getenv( "HOME", "/tmp" ) ) - except IOError: - pass - -import sys -try: - import rlcompleter, readline -except ImportError: - pass -else: - __registerExitHandler() - __enableReadlineSupport() diff --git a/meta/recipes-devtools/python/python3/support_SOURCE_DATE_EPOCH_in_py_compile.patch b/meta/recipes-devtools/python/python3/support_SOURCE_DATE_EPOCH_in_py_compile.patch deleted file mode 100644 index 32ecab9fec9..00000000000 --- a/meta/recipes-devtools/python/python3/support_SOURCE_DATE_EPOCH_in_py_compile.patch +++ /dev/null @@ -1,97 +0,0 @@ -The compiled .pyc files contain time stamp corresponding to the compile time. -This prevents binary reproducibility. This patch allows to achieve binary -reproducibility by overriding the build time stamp by the value -exported via SOURCE_DATE_EPOCH. - -Upstream-Status: Backport - -Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> - - -From aeab488630fdb1b56a8d0b0c13fa88706b2afe9b Mon Sep 17 00:00:00 2001 -From: "Bernhard M. Wiedemann" <bwiedemann@suse.de> -Date: Sat, 25 Feb 2017 06:42:28 +0100 -Subject: [PATCH] bpo-29708: support SOURCE_DATE_EPOCH env var in py_compile - -to allow for reproducible builds of python packages - -See https://reproducible-builds.org/ for why this is good -and https://reproducible-builds.org/specs/source-date-epoch/ -for the definition of this variable. - -Background: -In some distributions like openSUSE, binary rpms contain precompiled .pyc files. - -And packages like amqp or twisted dynamically generate .py files at build time -so those have the current time and that timestamp gets embedded -into the .pyc file header. -When we then adapt file timestamps in rpms to be constant, -the timestamp in the .pyc header will no more match -the .py timestamp in the filesystem. -The software will still work, but it will not use the .pyc file as it should. ---- - Doc/library/py_compile.rst | 4 ++++ - Lib/py_compile.py | 4 ++++ - Lib/test/test_py_compile.py | 19 +++++++++++++++++++ - 3 files changed, 27 insertions(+) - -diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst -index 0af8fb1..841f3e8 100644 ---- a/Doc/library/py_compile.rst -+++ b/Doc/library/py_compile.rst -@@ -53,6 +53,10 @@ byte-code cache files in the directory containing the source code. - :func:`compile` function. The default of ``-1`` selects the optimization - level of the current interpreter. - -+ If the SOURCE_DATE_EPOCH environment variable is set, the .py file mtime -+ and timestamp entry in .pyc file header, will be limited to this value. -+ See https://reproducible-builds.org/specs/source-date-epoch/ for more info. -+ - .. versionchanged:: 3.2 - Changed default value of *cfile* to be :PEP:`3147`-compliant. Previous - default was *file* + ``'c'`` (``'o'`` if optimization was enabled). -diff --git a/Lib/py_compile.py b/Lib/py_compile.py -index 11c5b50..62dcdc7 100644 ---- a/Lib/py_compile.py -+++ b/Lib/py_compile.py -@@ -137,6 +137,10 @@ def compile(file, cfile=None, dfile=None, doraise=False, optimize=-1): - except FileExistsError: - pass - source_stats = loader.path_stats(file) -+ sde = os.environ.get('SOURCE_DATE_EPOCH') -+ if sde and source_stats['mtime'] > int(sde): -+ source_stats['mtime'] = int(sde) -+ os.utime(file, (source_stats['mtime'], source_stats['mtime'])) - bytecode = importlib._bootstrap_external._code_to_bytecode( - code, source_stats['mtime'], source_stats['size']) - mode = importlib._bootstrap_external._calc_mode(file) -diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py -index 4a6caa5..3d09963 100644 ---- a/Lib/test/test_py_compile.py -+++ b/Lib/test/test_py_compile.py -@@ -98,6 +98,25 @@ def test_bad_coding(self): - self.assertFalse(os.path.exists( - importlib.util.cache_from_source(bad_coding))) - -+ def test_source_date_epoch(self): -+ testtime = 123456789 -+ orig_sde = os.getenv("SOURCE_DATE_EPOCH") -+ os.environ["SOURCE_DATE_EPOCH"] = str(testtime) -+ py_compile.compile(self.source_path, self.pyc_path) -+ if orig_sde: -+ os.environ["SOURCE_DATE_EPOCH"] = orig_sde -+ else: -+ del os.environ["SOURCE_DATE_EPOCH"] -+ self.assertTrue(os.path.exists(self.pyc_path)) -+ self.assertFalse(os.path.exists(self.cache_path)) -+ statinfo = os.stat(self.source_path) -+ self.assertEqual(statinfo.st_mtime, testtime) -+ f = open(self.pyc_path, "rb") -+ f.read(4) -+ timebytes = f.read(4) # read timestamp from pyc header -+ f.close() -+ self.assertEqual(timebytes, (testtime).to_bytes(4, 'little')) -+ - @unittest.skipIf(sys.flags.optimize > 0, 'test does not work with -O') - def test_double_dot_no_clobber(self): - # http://bugs.python.org/issue22966 diff --git a/meta/recipes-devtools/python/python3/sysconfig.py-add-_PYTHON_PROJECT_SRC.patch b/meta/recipes-devtools/python/python3/sysconfig.py-add-_PYTHON_PROJECT_SRC.patch deleted file mode 100644 index c89d9a3d836..00000000000 --- a/meta/recipes-devtools/python/python3/sysconfig.py-add-_PYTHON_PROJECT_SRC.patch +++ /dev/null @@ -1,51 +0,0 @@ -From fc93b3cc23b4f9698ae92a42986dbb02b6f19588 Mon Sep 17 00:00:00 2001 -From: Robert Yang <liezhi.yang@windriver.com> -Date: Sun, 18 Jan 2015 06:29:50 -0800 -Subject: [PATCH] sysconfig.py: add _PYTHON_PROJECT_SRC - -python3 has introduced _PYTHON_PROJECT_BASE which is used for separate B -and S, but it doesn't work when compile Modules, the target python3 runs -python3-native's sysconfig to get srcdir which is the native's, there -would be errors when native's srcdir has been removed, add -_PYTHON_PROJECT_SRC to fix the problem. - -Upstream-Status: Pending - -Signed-off-by: Robert Yang <liezhi.yang@windriver.com> ---- - Lib/distutils/sysconfig.py | 5 ++++- - Lib/sysconfig.py | 5 ++++- - 2 files changed, 8 insertions(+), 2 deletions(-) - -Index: Python-3.5.3/Lib/distutils/sysconfig.py -=================================================================== ---- Python-3.5.3.orig/Lib/distutils/sysconfig.py -+++ Python-3.5.3/Lib/distutils/sysconfig.py -@@ -498,7 +498,10 @@ def get_config_vars(*args): - _config_vars['SO'] = SO - - # Always convert srcdir to an absolute path -- srcdir = _config_vars.get('srcdir', project_base) -+ if "_PYTHON_PROJECT_SRC" in os.environ: -+ srcdir = os.path.abspath(os.environ["_PYTHON_PROJECT_SRC"]) -+ else: -+ srcdir = _config_vars.get('srcdir', project_base) - if os.name == 'posix': - if python_build: - # If srcdir is a relative path (typically '.' or '..') -Index: Python-3.5.3/Lib/sysconfig.py -=================================================================== ---- Python-3.5.3.orig/Lib/sysconfig.py -+++ Python-3.5.3/Lib/sysconfig.py -@@ -544,7 +544,10 @@ def get_config_vars(*args): - _CONFIG_VARS['userbase'] = _getuserbase() - - # Always convert srcdir to an absolute path -- srcdir = _CONFIG_VARS.get('srcdir', _PROJECT_BASE) -+ if "_PYTHON_PROJECT_SRC" in os.environ: -+ srcdir = os.path.abspath(os.environ["_PYTHON_PROJECT_SRC"]) -+ else: -+ srcdir = _CONFIG_VARS.get('srcdir', _PROJECT_BASE) - if os.name == 'posix': - if _PYTHON_BUILD: - # If srcdir is a relative path (typically '.' or '..') diff --git a/meta/recipes-devtools/python/python3/sysroot-include-headers.patch b/meta/recipes-devtools/python/python3/sysroot-include-headers.patch deleted file mode 100644 index 785b5567f29..00000000000 --- a/meta/recipes-devtools/python/python3/sysroot-include-headers.patch +++ /dev/null @@ -1,35 +0,0 @@ -Dont search hardcoded paths, we might be doing a cross-build -Use '=' in-front to let compiler append sysroot, if it can - -Should fix things like - -configure:6972: arm-angstrom-linux-gnueabi-gcc -march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a8 -DNDEBUG -fno-inline -D__SOFTFP__ --sysroot=/build/v2013.06/build/tmp-angstrom_v2013_06-eglibc/sysroots/beaglebone -c -O2 -pipe -g -feliminate-unused-debug-types -I/usr/include/ncursesw conftest.c >&5 -cc1: warning: include location "/usr/include/ncursesw" is unsafe for cross-compilation [-Wpoison-system-directories] - - -Signed-off-by: Khem Raj -Upstream-Status: Pending - - -Index: Python-3.3.2/setup.py -=================================================================== ---- Python-3.3.2.orig/setup.py 2013-07-30 01:30:48.000000000 -0700 -+++ Python-3.3.2/setup.py 2013-07-30 01:41:11.697862723 -0700 -@@ -1210,7 +1210,7 @@ - panel_library = 'panel' - if curses_library == 'ncursesw': - curses_defines.append(('HAVE_NCURSESW', '1')) -- curses_includes.append('/usr/include/ncursesw') -+ curses_includes.append('=/usr/include/ncursesw') - # Bug 1464056: If _curses.so links with ncursesw, - # _curses_panel.so must link with panelw. - panel_library = 'panelw' -@@ -1819,7 +1819,7 @@ - if host_platform == 'darwin': - # OS X 10.5 comes with libffi.dylib; the include files are - # in /usr/include/ffi -- inc_dirs.append('/usr/include/ffi') -+ inc_dirs.append('=/usr/include/ffi') - - ffi_inc = [sysconfig.get_config_var("LIBFFI_INCLUDEDIR")] - if not ffi_inc or ffi_inc[0] == '': diff --git a/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch b/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch deleted file mode 100644 index 8d037278bd3..00000000000 --- a/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch +++ /dev/null @@ -1,52 +0,0 @@ -From 5d6509313198ec9b686cad50b002212e4344004b Mon Sep 17 00:00:00 2001 -From: Hongxu Jia <hongxu.jia@windriver.com> -Date: Fri, 11 Mar 2016 01:15:45 -0500 -Subject: [PATCH] configure.ac: tweak MULTIARCH for powerpc-linux-gnuspe - -For p1022ds bsp, the MULTIARCH is powerpc-linux-gnuspev1 and -python3 did not recognize the extra 'v1' which caused python3 -configure error for the platform triplet. - -Q:Why Python3 check platform triplet? - -A:Under Linux, GNU/KFreeBSD and the Hurd, C extensions now include -the architecture triplet in the extension name, to make it easy to -test builds for different ABIs in the same working tree. - -Here is the generated C extensions which take platform triplet into account. -... -|image/usr/lib/python3.5/lib-dynload/_datetime.cpython-35m-powerpc-linux-gnuspe.so -... - -https://bugs.python.org/issue22980 -https://www.python.org/dev/peps/pep-3149/ -https://bugs.python.org/review/22980/patch/14593/54808 - -Upstream-Status: Pending - -Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> ---- - configure.ac | 7 +++++++ - 1 file changed, 7 insertions(+) - -diff --git a/configure.ac b/configure.ac -index 9eb3d22..c34a9a0 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -724,6 +724,13 @@ fi - - - MULTIARCH=$($CC --print-multiarch 2>/dev/null) -+ -+# Tweak MULTIARCH -+if test x$MULTIARCH = xpowerpc-linux-gnuspev1 -+then -+ MULTIARCH="powerpc-linux-gnuspe" -+fi -+ - AC_SUBST(MULTIARCH) - - AC_MSG_CHECKING([for the platform triplet based on compiler characteristics]) --- -1.9.1 - diff --git a/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-musl.patch b/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-musl.patch deleted file mode 100644 index 34c91755ff1..00000000000 --- a/meta/recipes-devtools/python/python3/tweak-MULTIARCH-for-powerpc-linux-musl.patch +++ /dev/null @@ -1,40 +0,0 @@ -From 7362464383bbd54f8e6be4389f2c74c0717bc6e1 Mon Sep 17 00:00:00 2001 -From: Serhey Popovych <serhe.popovych@gmail.com> -Date: Sat, 8 Dec 2018 11:24:06 -0500 -Subject: [PATCH] configure.ac: tweak MULTIARCH for powerpc-linux-musl - -For musl builds, the MULTIARCH is powerpc-linux-musl and configure.ac -does not have lines to recognize it causing configure error for the -platform triplet when building for powerpc 32bit. - -This is feature described in PEP-3149 and to prevent possible runtime -compatibility issues we map powerpc-linux-musl to powerpc-linux-gnu. - -Look at similar change by Hongxu Jia <hongxu.jia@> presended in -tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch to handle p1022ds BSP -builds. - -Upstream-Status: Pending -Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com> ---- - configure.ac | 4 ++++ - 1 file changed, 4 insertions(+) - -diff --git a/configure.ac b/configure.ac -index 95c98d1..1a4d8aa 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -752,6 +752,10 @@ if test x$MULTIARCH = xpowerpc-linux-gnuspev1 - then - MULTIARCH="powerpc-linux-gnuspe" - fi -+if test x$MULTIARCH = xpowerpc-linux-musl -+then -+ MULTIARCH="powerpc-linux-gnu" -+fi - - AC_SUBST(MULTIARCH) - --- -2.7.4 - diff --git a/meta/recipes-devtools/python/python3/unixccompiler.patch b/meta/recipes-devtools/python/python3/unixccompiler.patch deleted file mode 100644 index 3e2b1d1c2eb..00000000000 --- a/meta/recipes-devtools/python/python3/unixccompiler.patch +++ /dev/null @@ -1,35 +0,0 @@ -Upstream-Status: Pending - -The CC variable,sometimes like:"x86_64-poky-linux-gcc -m64 --sysroot=/${TMPDIR}/sysroots/qemux86-64", contains option information. -This will lead to wrong compiler name "qemux86-64" rather than "x86_64-poky-linux-gcc" when python finding the compiler name. - -Secondly add -L=<path> this way linker will be able to resolve /usr/lib w.r.t sysroot and not -use hardcoded /usr/lib to look for libs which is wrong in cross compile environment and this will work -ok on native systems too since sysroot for native compilers is / - -Signed-off-by: Mei Lei <lei.mei@intel.com> -Signed-off-by: Khem Raj <raj.khem@gmail.com> -Index: Python-3.3.2/Lib/distutils/unixccompiler.py -=================================================================== ---- Python-3.3.2.orig/Lib/distutils/unixccompiler.py 2013-05-15 09:32:54.000000000 -0700 -+++ Python-3.3.2/Lib/distutils/unixccompiler.py 2013-08-01 00:58:18.629056286 -0700 -@@ -202,7 +202,9 @@ - # ccompiler.py. - - def library_dir_option(self, dir): -- return "-L" + dir -+ if dir.startswith("/"): -+ return "-L=" + dir -+ return "-L" + dir - - def _is_gcc(self, compiler_name): - return "gcc" in compiler_name or "g++" in compiler_name -@@ -221,7 +221,7 @@ - # this time, there's no way to determine this information from - # the configuration data stored in the Python installation, so - # we use this hack. -- compiler = os.path.basename(sysconfig.get_config_var("CC")) -+ compiler = sysconfig.get_config_var("CC") - if sys.platform[:6] == "darwin": - # MacOSX's linker doesn't understand the -R flag at all - return "-L" + dir diff --git a/meta/recipes-devtools/python/python3_3.5.6.bb b/meta/recipes-devtools/python/python3_3.5.6.bb deleted file mode 100644 index d64cb18c383..00000000000 --- a/meta/recipes-devtools/python/python3_3.5.6.bb +++ /dev/null @@ -1,334 +0,0 @@ -require recipes-devtools/python/python3.inc - -DEPENDS = "python3-native libffi bzip2 gdbm openssl \ - sqlite3 zlib virtual/libintl xz qemu-native \ - qemu-helper-native virtual/crypt" - -DISTRO_SRC_URI ?= "file://sitecustomize.py" -DISTRO_SRC_URI_linuxstdbase = "" -SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ -file://python-config.patch \ -file://030-fixup-include-dirs.patch \ -file://130-readline-setup.patch \ -file://150-fix-setupterm.patch \ -file://0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch \ -file://tweak-MULTIARCH-for-powerpc-linux-gnuspe.patch \ -file://tweak-MULTIARCH-for-powerpc-linux-musl.patch \ -file://support_SOURCE_DATE_EPOCH_in_py_compile.patch \ -${DISTRO_SRC_URI} \ -" - -SRC_URI += "\ - file://03-fix-tkinter-detection.patch \ - ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \ - file://cgi_py.patch \ - file://host_include_contamination.patch \ - file://python-3.3-multilib.patch \ - file://sysroot-include-headers.patch \ - file://unixccompiler.patch \ - file://avoid-ncursesw-include-path.patch \ - file://python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \ - file://sysconfig.py-add-_PYTHON_PROJECT_SRC.patch \ - file://setup.py-check-cross_compiling-when-get-FLAGS.patch \ - file://configure.ac-fix-LIBPL.patch \ - file://0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch \ - file://pass-missing-libraries-to-Extension-for-mul.patch \ - file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \ - file://0002-Makefile-add-target-to-split-profile-generation.patch \ - file://float-endian.patch \ - file://ftplib.patch \ - file://signal.patch \ - file://0001-Issue-28043-SSLContext-has-improved-default-settings.patch \ - file://0002-bpo-29136-Add-TLS-1.3-cipher-suites-and-OP_NO_TLSv1_.patch \ - file://0003-bpo-32947-Fixes-for-TLS-1.3-and-OpenSSL-1.1.1-GH-876.patch \ - file://0004-bpo-33570-TLS-1.3-ciphers-for-OpenSSL-1.1.1-GH-6976.patch \ - file://0005-bpo-30714-ALPN-changes-for-OpenSSL-1.1.0f-2305.patch \ - file://run-ptest \ - file://0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch \ - " - -inherit multilib_header python3native update-alternatives qemu ptest - -MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}" - -ALTERNATIVE_${PN}-dev = "python-config" -ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" -ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" - -CONFIGUREOPTS += " --with-system-ffi " - -CACHED_CONFIGUREVARS = "ac_cv_have_chflags=no \ - ac_cv_have_lchflags=no \ - ac_cv_have_long_long_format=yes \ - ac_cv_buggy_getaddrinfo=no \ - ac_cv_file__dev_ptmx=yes \ - ac_cv_file__dev_ptc=no \ -" - -TARGET_CC_ARCH += "-DNDEBUG -fno-inline" -SDK_CC_ARCH += "-DNDEBUG -fno-inline" -EXTRA_OEMAKE += "CROSS_COMPILE=yes" -EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip" - -PYTHON3_PROFILE_TASK ?= "./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true" - -export CROSS_COMPILE = "${TARGET_PREFIX}" -export CCSHARED = "-fPIC" - -# Fix cross compilation of different modules -export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/:${B}/build/lib.linux-${TARGET_ARCH}-${PYTHON_MAJMIN}:${S}/Lib:${S}/Lib/plat-linux" - -PACKAGECONFIG ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" -PACKAGECONFIG[readline] = ",,readline" -# Use profile guided optimisation by running PyBench inside qemu-user -PACKAGECONFIG[pgo] = "--enable-optimizations" -PACKAGECONFIG[tk] = ",,tk" - -do_configure_append() { - rm -f ${S}/Makefile.orig -} - -run_make() { - oe_runmake PGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ - HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ - OPT="${CFLAGS}" \ - "$@" -} - -do_compile() { - # regenerate platform specific files, because they depend on system headers - cd ${S}/Lib/plat-linux* - include=${STAGING_INCDIR} ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - ${S}/Tools/scripts/h2py.py -i '(u_long)' \ - ${STAGING_INCDIR}/dlfcn.h \ - ${STAGING_INCDIR}/linux/cdrom.h \ - ${STAGING_INCDIR}/netinet/in.h \ - ${STAGING_INCDIR}/sys/types.h - sed -e 's,${STAGING_DIR_HOST},,g' -i *.py - cd - - - # remove any bogus LD_LIBRARY_PATH - sed -i -e s,RUNSHARED=.*,RUNSHARED=, Makefile - - if [ ! -f Makefile.orig ]; then - install -m 0644 Makefile Makefile.orig - fi - sed -i -e 's,^CONFIGURE_LDFLAGS=.*,CONFIGURE_LDFLAGS=-L. -L${STAGING_LIBDIR},g' \ - -e 's,libdir=${libdir},libdir=${STAGING_LIBDIR},g' \ - -e 's,libexecdir=${libexecdir},libexecdir=${STAGING_DIR_HOST}${libexecdir},g' \ - -e 's,^LIBDIR=.*,LIBDIR=${STAGING_LIBDIR},g' \ - -e 's,includedir=${includedir},includedir=${STAGING_INCDIR},g' \ - -e 's,^INCLUDEDIR=.*,INCLUDE=${STAGING_INCDIR},g' \ - -e 's,^CONFINCLUDEDIR=.*,CONFINCLUDE=${STAGING_INCDIR},g' \ - Makefile - # save copy of it now, because if we do it in do_install and - # then call do_install twice we get Makefile.orig == Makefile.sysroot - install -m 0644 Makefile Makefile.sysroot - - if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then - run_make profile-opt - qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" - cat >pgo-wrapper <<EOF -#!/bin/sh -cd ${B} -$qemu_binary "\$@" -EOF - chmod +x pgo-wrapper - bbnote Gathering profiling data - ./pgo-wrapper ${PYTHON3_PROFILE_TASK} - bbnote Profiling data gathered, rebuilding - run_make clean_and_use_profile - else - run_make libpython3.so - run_make - fi -} - -do_install() { - # make install needs the original Makefile, or otherwise the inclues would - # go to ${D}${STAGING...}/... - install -m 0644 Makefile.orig Makefile - - install -d ${D}${libdir}/pkgconfig - install -d ${D}${libdir}/python${PYTHON_MAJMIN}/config - - # rerun the build once again with original makefile this time - # run install in a separate step to avoid compile/install race - if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then - run_make DESTDIR=${D} LIBDIR=${libdir} build_all_use_profile - else - run_make DESTDIR=${D} LIBDIR=${libdir} - fi - - run_make DESTDIR=${D} LIBDIR=${libdir} install - - # avoid conflict with 2to3 from Python 2 - rm -f ${D}/${bindir}/2to3 - - install -m 0644 Makefile.sysroot ${D}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile - install -m 0644 Makefile.sysroot ${D}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}/Makefile - - if [ -e ${WORKDIR}/sitecustomize.py ]; then - install -m 0644 ${WORKDIR}/sitecustomize.py ${D}/${libdir}/python${PYTHON_MAJMIN} - fi - - oe_multilib_header python${PYTHON_BINABI}/pyconfig.h -} - -do_install_append_class-nativesdk () { - create_wrapper ${D}${bindir}/python${PYTHON_MAJMIN} PYTHONHOME='${prefix}' TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' PYTHONNOUSERSITE='1' -} - -SSTATE_SCAN_FILES += "Makefile" -PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" - -py_package_preprocess () { - # copy back the old Makefile to fix target package - install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile - install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}/Makefile - # Remove references to buildmachine paths in target Makefile and _sysconfigdata - sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ - -e 's|${DEBUG_PREFIX_MAP}||g' \ - -e 's:${HOSTTOOLS_DIR}/::g' \ - -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ - -e 's:${RECIPE_SYSROOT}::g' \ - -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ - ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \ - ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}/Makefile \ - ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py \ - ${PKGD}/${bindir}/python${PYTHON_BINABI}-config - - # Recompile _sysconfigdata after modifying it - cd ${PKGD} - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} \ - -c "from py_compile import compile; compile('./${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py')" - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} \ - -c "from py_compile import compile; compile('./${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py', optimize=1)" - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} \ - -c "from py_compile import compile; compile('./${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py', optimize=2)" - cd - - - mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} -} - -PACKAGES_remove = "${PN}" - -# manual dependency additions -RPROVIDES_${PN}-modules = "${PN}" -RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" -RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" - -FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" -FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" -FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" - -# provide python-pyvenv from python3-venv -RPROVIDES_${PN}-venv += "${PN}-pyvenv" - -# package libpython3 -PACKAGES =+ "libpython3 libpython3-staticdev" -FILES_libpython3 = "${libdir}/libpython*.so.*" -FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}/libpython${PYTHON_BINABI}.a" -INSANE_SKIP_${PN}-dev += "dev-elf" - -# catch all the rest (unsorted) -PACKAGES += "${PN}-misc" -RDEPENDS_${PN}-misc += "${PN}-core ${PN}-email ${PN}-codecs" -RDEPENDS_${PN}-modules += "${PN}-misc" -FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" - -# catch manpage -PACKAGES += "${PN}-man" -FILES_${PN}-man = "${datadir}/man" - -RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" -RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" -RDEPENDS_${PN}-dev = "" - -BBCLASSEXTEND = "nativesdk" - -# We want bytecode precompiled .py files (.pyc's) by default -# but the user may set it on their own conf -INCLUDE_PYCS ?= "1" - -python(){ - import collections, json - - filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') - # This python changes the datastore based on the contents of a file, so mark - # that dependency. - bb.parse.mark_dependency(d, filename) - - with open(filename) as manifest_file: - manifest_str = manifest_file.read() - json_start = manifest_str.find('# EOC') + 6 - manifest_file.seek(json_start) - manifest_str = manifest_file.read() - python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) - - include_pycs = d.getVar('INCLUDE_PYCS') - - packages = d.getVar('PACKAGES').split() - pn = d.getVar('PN') - - newpackages=[] - for key in python_manifest: - pypackage= pn + '-' + key - - if pypackage not in packages: - # We need to prepend, otherwise python-misc gets everything - # so we use a new variable - newpackages.append(pypackage) - - # "Build" python's manifest FILES, RDEPENDS and SUMMARY - d.setVar('FILES_' + pypackage, '') - for value in python_manifest[key]['files']: - d.appendVar('FILES_' + pypackage, ' ' + value) - - # Add cached files - if include_pycs == '1': - for value in python_manifest[key]['cached']: - d.appendVar('FILES_' + pypackage, ' ' + value) - - for value in python_manifest[key]['rdepends']: - # Make it work with or without $PN - if '${PN}' in value: - value=value.split('-')[1] - d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) - d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) - - # Prepending so to avoid python-misc getting everything - packages = newpackages + packages - d.setVar('PACKAGES', ' '.join(packages)) - d.setVar('ALLOW_EMPTY_${PN}-modules', '1') -} - -# Files needed to create a new manifest -SRC_URI += "file://create_manifest3.py file://get_module_deps3.py file://python3-manifest.json" - -do_create_manifest() { - # This task should be run with every new release of Python. - # We must ensure that PACKAGECONFIG enables everything when creating - # a new manifest, this is to base our new manifest on a complete - # native python build, containing all dependencies, otherwise the task - # wont be able to find the required files. - # e.g. BerkeleyDB is an optional build dependency so it may or may not - # be present, we must ensure it is. - - cd ${WORKDIR} - # This needs to be executed by python-native and NOT by HOST's python - nativepython3 create_manifest3.py ${PYTHON_MAJMIN} - cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json -} - -# bitbake python -c create_manifest -addtask do_create_manifest - -# Make sure we have native python ready when we create a new manifest -do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" -do_create_manifest[depends] += "python3:do_patch" -- 2.17.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/3] lib/oeqa/runtime/cases/gi.py: fix deprecation warning 2019-02-01 12:58 [PATCH 1/3] python3: upgrade to 3.7.2 Alexander Kanavin 2019-02-01 12:58 ` [PATCH 2/3] python3: delete old 3.5.6 version Alexander Kanavin @ 2019-02-01 12:58 ` Alexander Kanavin 2019-02-01 20:49 ` [PATCH 1/3] python3: upgrade to 3.7.2 Khem Raj 2 siblings, 0 replies; 8+ messages in thread From: Alexander Kanavin @ 2019-02-01 12:58 UTC (permalink / raw) To: openembedded-core With newest Python, the following is printed: PyGIDeprecationWarning: GObject.markup_escape_text is deprecated; use GLib.markup_escape_text instead) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> --- meta/lib/oeqa/runtime/cases/gi.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/runtime/cases/gi.py b/meta/lib/oeqa/runtime/cases/gi.py index 19073e52cba..7e16651df74 100644 --- a/meta/lib/oeqa/runtime/cases/gi.py +++ b/meta/lib/oeqa/runtime/cases/gi.py @@ -9,7 +9,7 @@ class GObjectIntrospectionTest(OERuntimeTestCase): @OETestDepends(["ssh.SSHTest.test_ssh"]) @OEHasPackage(["python3-pygobject"]) def test_python(self): - script = """from gi.repository import GObject; print(GObject.markup_escape_text("<testing&testing>"))""" + script = """from gi.repository import GLib; print(GLib.markup_escape_text("<testing&testing>"))""" status, output = self.target.run("python3 -c '%s'" % script) self.assertEqual(status, 0, msg="Python failed (%s)" % (output)) self.assertEqual(output, "<testing&testing>", msg="Unexpected output (%s)" % output) -- 2.17.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] python3: upgrade to 3.7.2 2019-02-01 12:58 [PATCH 1/3] python3: upgrade to 3.7.2 Alexander Kanavin 2019-02-01 12:58 ` [PATCH 2/3] python3: delete old 3.5.6 version Alexander Kanavin 2019-02-01 12:58 ` [PATCH 3/3] lib/oeqa/runtime/cases/gi.py: fix deprecation warning Alexander Kanavin @ 2019-02-01 20:49 ` Khem Raj 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego 2 siblings, 1 reply; 8+ messages in thread From: Khem Raj @ 2019-02-01 20:49 UTC (permalink / raw) To: Alexander Kanavin; +Cc: Patches and discussions about the oe-core layer On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > I took the same approach as the recent perl upgrade: write recipe from scratch, > taking the pieces from the old recipe only when they were proven to be necessary. > > The pgo, manifest and ptest features are all preserved. > > New features: > > - native and target recipes are now unified into one recipe > > - check_build_completeness.py runs right after do_compile() and verifies that > all optional modules have been built (a notorious source of regressions) > > - a new approach to sysconfig.py and distutils/sysconfig.py returning values > appropriate for native or target builds: we copy the configuration file to a > separate folder, add that folder to sys.path (through environment variable > that differs between native and target builds), and point python to the file > through another environment variable. > Thanks, once current cycle is over for OE, I would like to give this a whirl and see how it goes > Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > --- > meta/classes/python3-dir.bbclass | 2 +- > meta/classes/python3native.bbclass | 2 + > ...ib-termcap-to-linker-flags-to-avoid-.patch | 25 + > ...hell-version-of-python-config-that-w.patch | 35 + > ...-qemu-wrapper-when-gathering-profile.patch | 25 + > ...fig-append-STAGING_LIBDIR-python-sys.patch | 42 + > ...tutils-prefix-is-inside-staging-area.patch | 54 + > .../python3/avoid_warning_about_tkinter.patch | 36 + > .../python-sanity/python3/cgi_py.patch | 32 + > .../python3/check_build_completeness.py | 17 + > .../python-sanity/python3/create_manifest3.py | 433 ++++++ > .../python-sanity/python3/get_module_deps3.py | 146 ++ > .../python-sanity/python3/python-config.patch | 46 + > .../python3/python3-manifest.json | 1227 +++++++++++++++++ > .../python-sanity/python3/run-ptest | 3 + > .../python-sanity/python3_3.7.2.bb | 281 ++++ > 16 files changed, 2405 insertions(+), 1 deletion(-) > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch > create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json > create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest > create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb > > diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass > index 06bb046d9c2..7dd130bad99 100644 > --- a/meta/classes/python3-dir.bbclass > +++ b/meta/classes/python3-dir.bbclass > @@ -1,4 +1,4 @@ > -PYTHON_BASEVERSION = "3.5" > +PYTHON_BASEVERSION = "3.7" > PYTHON_ABI = "m" > PYTHON_DIR = "python${PYTHON_BASEVERSION}" > PYTHON_PN = "python3" > diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass > index da12a714703..a3acaf61bbd 100644 > --- a/meta/classes/python3native.bbclass > +++ b/meta/classes/python3native.bbclass > @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native " > export STAGING_INCDIR > export STAGING_LIBDIR > > +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" > + > # suppress host user's site-packages dirs. > export PYTHONNOUSERSITE = "1" > > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > new file mode 100644 > index 00000000000..09f279ba1d7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > @@ -0,0 +1,25 @@ > +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Fri, 25 Jan 2019 19:04:13 +0100 > +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host > + contamination > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > + > +--- > + setup.py | 1 - > + 1 file changed, 1 deletion(-) > + > +diff --git a/setup.py b/setup.py > +index b4357e3..fbec00d 100644 > +--- a/setup.py > ++++ b/setup.py > +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext): > + 'termcap'): > + readline_libs.append('termcap') > + exts.append( Extension('readline', ['readline.c'], > +- library_dirs=['/usr/lib/termcap'], > + extra_link_args=readline_extra_link_args, > + libraries=readline_libs) ) > + else: > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > new file mode 100644 > index 00000000000..83fd52d87f4 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > @@ -0,0 +1,35 @@ > +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Tue, 29 Jan 2019 15:03:01 +0100 > +Subject: [PATCH] Do not use the shell version of python-config that was > + introduced in 3.4 > + > +Revert instead to the original python version: it has our tweaks and > +outputs directories correctly. > + > +Upstream-Status: Inappropriate [oe-specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > +--- > + Makefile.pre.in | 9 +++------ > + 1 file changed, 3 insertions(+), 6 deletions(-) > + > +diff --git a/Makefile.pre.in b/Makefile.pre.in > +index 2d2e11f..cc19942 100644 > +--- a/Makefile.pre.in > ++++ b/Makefile.pre.in > +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh > + sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py > + @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} > + LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config > +- @ # On Darwin, always use the python version of the script, the shell > +- @ # version doesn't use the compiler customizations that are provided > +- @ # in python (_osx_support.py). > +- @if test `uname -s` = Darwin; then \ > +- cp python-config.py python-config; \ > +- fi > ++ @ # In OpenEmbedded, always use the python version of the script, the shell > ++ @ # version is broken in multiple ways, and doesn't return correct directories > ++ cp python-config.py python-config > + > + > + # Install the include files > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > new file mode 100644 > index 00000000000..fa7735ff93e > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > @@ -0,0 +1,25 @@ > +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Wed, 30 Jan 2019 12:41:04 +0100 > +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > +--- > + Makefile.pre.in | 3 +-- > + 1 file changed, 1 insertion(+), 2 deletions(-) > + > +diff --git a/Makefile.pre.in b/Makefile.pre.in > +index a3a02a7..d5503dd 100644 > +--- a/Makefile.pre.in > ++++ b/Makefile.pre.in > +@@ -507,8 +507,7 @@ build_all_generate_profile: > + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" > + > + run_profile_task: > +- @ # FIXME: can't run for a cross build > +- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true > ++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true > + > + build_all_merge_profile: > + $(LLVM_PROF_MERGER) > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > new file mode 100644 > index 00000000000..2c5b76f131c > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > @@ -0,0 +1,42 @@ > +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Thu, 31 Jan 2019 16:46:30 +0100 > +Subject: [PATCH] distutils/sysconfig: append > + STAGING_LIBDIR/python-sysconfigdata to sys.path > + > +So that target configuration can be used when running native python > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > + > +--- > + Lib/distutils/sysconfig.py | 2 ++ > + Lib/sysconfig.py | 2 ++ > + 2 files changed, 4 insertions(+) > + > +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py > +index e07a6c8..6b8c129 100644 > +--- a/Lib/distutils/sysconfig.py > ++++ b/Lib/distutils/sysconfig.py > +@@ -421,6 +421,8 @@ def _init_posix(): > + platform=sys.platform, > + multiarch=getattr(sys.implementation, '_multiarch', ''), > + )) > ++ if 'STAGING_LIBDIR' in os.environ: > ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') > + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > + build_time_vars = _temp.build_time_vars > + global _config_vars > +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py > +index 9ee4d31..e586abd 100644 > +--- a/Lib/sysconfig.py > ++++ b/Lib/sysconfig.py > +@@ -412,6 +412,8 @@ def _init_posix(vars): > + """Initialize the module as appropriate for POSIX systems.""" > + # _sysconfigdata is generated at build time, see _generate_posix_vars() > + name = _get_sysconfigdata_name() > ++ if 'STAGING_LIBDIR' in os.environ: > ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') > + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > + build_time_vars = _temp.build_time_vars > + vars.update(build_time_vars) > diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > new file mode 100644 > index 00000000000..abf08f59dd7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > @@ -0,0 +1,54 @@ > +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001 > +From: Khem Raj <raj.khem@gmail.com> > +Date: Tue, 14 May 2013 15:00:26 -0700 > +Subject: [PATCH] python3: Add target and native recipes > + > +Upstream-Status: Inappropriate [embedded specific] > + > +02/2015 Rebased for Python 3.4.2 > + > +# The proper prefix is inside our staging area. > +# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de> > +# Signed-off-by: Phil Blundell <philb@gnu.org> > +# Signed-off-by: Khem Raj <raj.khem@gmail.com> > +# Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> > + > +--- > + Lib/distutils/sysconfig.py | 10 ++++++++-- > + 1 file changed, 8 insertions(+), 2 deletions(-) > + > +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py > +index 6b8c129..3ca7f79 100644 > +--- a/Lib/distutils/sysconfig.py > ++++ b/Lib/distutils/sysconfig.py > +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): > + If 'prefix' is supplied, use it instead of sys.base_prefix or > + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. > + """ > +- if prefix is None: > ++ if prefix is None and os.environ['STAGING_INCDIR'] != "": > ++ prefix = os.environ['STAGING_INCDIR'].rstrip('include') > ++ elif prefix is None: > + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX > + if os.name == "posix": > + if python_build: > +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): > + If 'prefix' is supplied, use it instead of sys.base_prefix or > + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. > + """ > ++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] > ++ if prefix is None and os.environ['STAGING_LIBDIR'] != "": > ++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) > ++ > + if prefix is None: > + if standard_lib: > + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX > +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): > + > + if os.name == "posix": > + libpython = os.path.join(prefix, > +- "lib", "python" + get_python_version()) > ++ lib_basename, "python" + get_python_version()) > + if standard_lib: > + return libpython > + else: > diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > new file mode 100644 > index 00000000000..24e67b4ca14 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > @@ -0,0 +1,36 @@ > +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001 > +From: Andrei Gherzan <andrei@gherzan.ro> > +Date: Mon, 28 Jan 2019 15:57:54 +0000 > +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet > + integrated in yocto so we skip the check for this module. Avoid a warning by > + not adding this module to missing variable. > + > +Upstream-Status: Inappropriate [distribution] > + > +Also simply disable the tk module since its not in DEPENDS. > +Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> > + > +--- > + setup.py | 8 +++++--- > + 1 file changed, 5 insertions(+), 3 deletions(-) > + > +diff --git a/setup.py b/setup.py > +index fbec00d..b7a36a6 100644 > +--- a/setup.py > ++++ b/setup.py > +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext): > + self.extensions.extend(exts) > + > + # Call the method for detecting whether _tkinter can be compiled > +- self.detect_tkinter(inc_dirs, lib_dirs) > ++ # self.detect_tkinter(inc_dirs, lib_dirs) > + > +- if '_tkinter' not in [e.name for e in self.extensions]: > +- missing.append('_tkinter') > ++ # tkinter module will not be avalaible as yocto > ++ # doesn't have tk integrated (yet) > ++ #if '_tkinter' not in [e.name for e in self.extensions]: > ++ # missing.append('_tkinter') > + > + # Build the _uuid module if possible > + uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"]) > diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch > new file mode 100644 > index 00000000000..6c4ba54320b > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch > @@ -0,0 +1,32 @@ > +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001 > +From: Mark Hatle <mark.hatle@windriver.com> > +Date: Wed, 21 Sep 2011 20:55:33 -0500 > +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment > + > +Upstream-Status: Inappropriate [distribution] > + > +Signed-off-by: Mark Hatle <mark.hatle@windriver.com> > + > +--- > + Lib/cgi.py | 11 +---------- > + 1 file changed, 1 insertion(+), 10 deletions(-) > + > +diff --git a/Lib/cgi.py b/Lib/cgi.py > +index 8cf6687..094c7b4 100755 > +--- a/Lib/cgi.py > ++++ b/Lib/cgi.py > +@@ -1,13 +1,4 @@ > +-#! /usr/local/bin/python > +- > +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is > +-# intentionally NOT "/usr/bin/env python". On many systems > +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI > +-# scripts, and /usr/local/bin is the default directory where Python is > +-# installed, so /usr/bin/env would be unable to find python. Granted, > +-# binary installations by Linux vendors often install Python in > +-# /usr/bin. So let those vendors patch cgi.py to match their choice > +-# of installation. > ++#! /usr/bin/env python > + > + """Support module for CGI (Common Gateway Interface) scripts. > + > diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > new file mode 100755 > index 00000000000..a1eace3f571 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > @@ -0,0 +1,17 @@ > +#!/usr/bin/env python3 > +import sys > +logfile = open(sys.argv[1]).read() > + > +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found") > +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.") > +if necessary_bits != -1: > + print("%s" %(logfile[necessary_bits:to_find_bits])) > + > +failed_to_build = logfile.find("Failed to build these modules:") > +if failed_to_build != -1: > + failed_to_build_end = logfile.find("\n\n", failed_to_build) > + print("%s" %(logfile[failed_to_build:failed_to_build_end])) > + > +if necessary_bits != -1 or failed_to_build != -1: > + sys.exit(1) > + > diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py > new file mode 100644 > index 00000000000..4da02a2991a > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py > @@ -0,0 +1,433 @@ > +# This script is used as a bitbake task to create a new python manifest > +# $ bitbake python -c create_manifest > +# > +# Our goal is to keep python-core as small as posible and add other python > +# packages only when the user needs them, hence why we split upstream python > +# into several packages. > +# > +# In a very simplistic way what this does is: > +# Launch python and see specifically what is required for it to run at a minimum > +# > +# Go through the python-manifest file and launch a separate task for every single > +# one of the files on each package, this task will check what was required for that > +# specific module to run, these modules will be called dependencies. > +# The output of such task will be a list of the modules or dependencies that were > +# found for that file. > +# > +# Such output will be parsed by this script, we will look for each dependency on the > +# manifest and if we find that another package already includes it, then we will add > +# that package as an RDEPENDS to the package we are currently checking; in case we dont > +# find the current dependency on any other package we will add it to the current package > +# as part of FILES. > +# > +# > +# This way we will create a new manifest from the data structure that was built during > +# this process, on this new manifest each package will contain specifically only > +# what it needs to run. > +# > +# There are some caveats which we try to deal with, such as repeated files on different > +# packages, packages that include folders, wildcards, and special packages. > +# Its also important to note that this method only works for python files, and shared > +# libraries. Static libraries, header files and binaries need to be dealt with manually. > +# > +# This script differs from its python2 version mostly on how shared libraries are handled > +# The manifest file for python3 has an extra field which contains the cached files for > +# each package. > +# Tha method to handle cached files does not work when a module includes a folder which > +# itself contains the pycache folder, gladly this is almost never the case. > +# > +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> > + > + > +import sys > +import subprocess > +import json > +import os > +import collections > + > +# Get python version from ${PYTHON_MAJMIN} > +pyversion = str(sys.argv[1]) > + > +# Hack to get native python search path (for folders), not fond of it but it works for now > +pivot = 'recipe-sysroot-native' > +for p in sys.path: > + if pivot in p: > + nativelibfolder = p[:p.find(pivot)+len(pivot)] > + > +# Empty dict to hold the whole manifest > +new_manifest = collections.OrderedDict() > + > +# Check for repeated files, folders and wildcards > +allfiles = [] > +repeated = [] > +wildcards = [] > + > +hasfolders = [] > +allfolders = [] > + > +def isFolder(value): > + value = value.replace('${PYTHON_MAJMIN}',pyversion) > + if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): > + return True > + else: > + return False > + > +def isCached(item): > + if '__pycache__' in item: > + return True > + else: > + return False > + > +def prepend_comments(comments, json_manifest): > + with open(json_manifest, 'r+') as manifest: > + json_contents = manifest.read() > + manifest.seek(0, 0) > + manifest.write(comments + json_contents) > + > +# Read existing JSON manifest > +with open('python3-manifest.json') as manifest: > + # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker > + manifest_str = manifest.read() > + json_start = manifest_str.find('# EOC') + 6 # EOC + \n > + manifest.seek(0) > + comments = manifest.read(json_start) > + manifest_str = manifest.read() > + old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) > + > +# > +# First pass to get core-package functionality, because we base everything on the fact that core is actually working > +# Not exactly the same so it should not be a function > +# > + > +print ('Getting dependencies for package: core') > + > + > +# This special call gets the core dependencies and > +# appends to the old manifest so it doesnt hurt what it > +# currently holds. > +# This way when other packages check for dependencies > +# on the new core package, they will still find them > +# even when checking the old_manifest > + > +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') > +for coredep in output.split(): > + coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') > + if isCached(coredep): > + if coredep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(coredep) > + else: > + if coredep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(coredep) > + > + > +# The second step is to loop through the existing files contained in the core package > +# according to the old manifest, identify if they are modules, or some other type > +# of file that we cant import (directories, binaries, configs) in which case we > +# can only assume they were added correctly (manually) so we ignore those and > +# pass them to the manifest directly. > + > +for filedep in old_manifest['core']['files']: > + if isFolder(filedep): > + if isCached(filedep): > + if filedep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(filedep) > + else: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + if '${bindir}' in filedep: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + if filedep == '': > + continue > + if '${includedir}' in filedep: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + > + # Get actual module name , shouldnt be affected by libdir/bindir, etc. > + pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] > + > + > + # We now know that were dealing with a python module, so we can import it > + # and check what its dependencies are. > + # We launch a separate task for each module for deterministic behavior. > + # Each module will only import what is necessary for it to work in specific. > + # The output of each task will contain each module's dependencies > + > + print ('Getting dependencies for module: %s' % pymodule) > + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') > + print ('The following dependencies were found for module %s:\n' % pymodule) > + print (output) > + > + > + for pymodule_dep in output.split(): > + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') > + > + if isCached(pymodule_dep): > + if pymodule_dep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(pymodule_dep) > + else: > + if pymodule_dep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(pymodule_dep) > + > + > +# At this point we are done with the core package. > +# The old_manifest dictionary is updated only for the core package because > +# all others will use this a base. > + > + > +# To improve the script speed, we check which packages contain directories > +# since we will be looping through (only) those later. > +for pypkg in old_manifest: > + for filedep in old_manifest[pypkg]['files']: > + if isFolder(filedep): > + print ('%s is a folder' % filedep) > + if pypkg not in hasfolders: > + hasfolders.append(pypkg) > + if filedep not in allfolders: > + allfolders.append(filedep) > + > + > + > +# This is the main loop that will handle each package. > +# It works in a similar fashion than the step before, but > +# we will now be updating a new dictionary that will eventually > +# become the new manifest. > +# > +# The following loops though all packages in the manifest, > +# through all files on each of them, and checks whether or not > +# they are modules and can be imported. > +# If they can be imported, then it checks for dependencies for > +# each of them by launching a separate task. > +# The output of that task is then parsed and the manifest is updated > +# accordingly, wether it should add the module on FILES for the current package > +# or if that module already belongs to another package then the current one > +# will RDEPEND on it > + > +for pypkg in old_manifest: > + # Use an empty dict as data structure to hold data for each package and fill it up > + new_manifest[pypkg] = collections.OrderedDict() > + new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] > + new_manifest[pypkg]['rdepends'] = [] > + new_manifest[pypkg]['files'] = [] > + new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] > + > + # All packages should depend on core > + if pypkg != 'core': > + new_manifest[pypkg]['rdepends'].append('core') > + new_manifest[pypkg]['cached'] = [] > + > + print('\n') > + print('--------------------------') > + print ('Handling package %s' % pypkg) > + print('--------------------------') > + > + # Handle special cases, we assume that when they were manually added > + # to the manifest we knew what we were doing. > + special_packages = ['misc', 'modules', 'dev', 'tests'] > + if pypkg in special_packages or 'staticdev' in pypkg: > + print('Passing %s package directly' % pypkg) > + new_manifest[pypkg] = old_manifest[pypkg] > + continue > + > + for filedep in old_manifest[pypkg]['files']: > + # We already handled core on the first pass, we can ignore it now > + if pypkg == 'core': > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + > + # Handle/ignore what we cant import > + if isFolder(filedep): > + new_manifest[pypkg]['files'].append(filedep) > + # Asyncio (and others) are both the package and the folder name, we should not skip those... > + path,mod = os.path.split(filedep) > + if mod != pypkg: > + continue > + if '${bindir}' in filedep: > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + if filedep == '': > + continue > + if '${includedir}' in filedep: > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + > + # Get actual module name , shouldnt be affected by libdir/bindir, etc. > + # We need to check if the imported module comes from another (e.g. sqlite3.dump) > + path,pymodule = os.path.split(filedep) > + path = os.path.basename(path) > + pymodule = os.path.splitext(os.path.basename(pymodule))[0] > + > + # If this condition is met, it means we need to import it from another module > + # or its the folder itself (e.g. unittest) > + if path == pypkg: > + if pymodule: > + pymodule = path + '.' + pymodule > + else: > + pymodule = path > + > + > + > + # We now know that were dealing with a python module, so we can import it > + # and check what its dependencies are. > + # We launch a separate task for each module for deterministic behavior. > + # Each module will only import what is necessary for it to work in specific. > + # The output of each task will contain each module's dependencies > + > + print ('\nGetting dependencies for module: %s' % pymodule) > + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') > + print ('The following dependencies were found for module %s:\n' % pymodule) > + print (output) > + > + reportFILES = [] > + reportRDEPS = [] > + > + for pymodule_dep in output.split(): > + > + # Warning: This first part is ugly > + # One of the dependencies that was found, could be inside of one of the folders included by another package > + # We need to check if this happens so we can add the package containing the folder as an rdependency > + # e.g. Folder encodings contained in codecs > + # This would be solved if no packages included any folders > + > + # This can be done in two ways: > + # 1 - We assume that if we take out the filename from the path we would get > + # the folder string, then we would check if folder string is in the list of folders > + # This would not work if a package contains a folder which contains another folder > + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 > + # folder_string would not match any value contained in the list of folders > + # > + # 2 - We do it the other way around, checking if the folder is contained in the path > + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 > + # is folder_string inside path/folder1/folder2/filename?, > + # Yes, it works, but we waste a couple of milliseconds. > + > + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') > + inFolders = False > + for folder in allfolders: > + # The module could have a directory named after it, e.g. xml, if we take out the filename from the path > + # we'll end up with ${libdir}, and we want ${libdir}/xml > + if isFolder(pymodule_dep): > + check_path = pymodule_dep > + else: > + check_path = os.path.dirname(pymodule_dep) > + if folder in check_path : > + inFolders = True # Did we find a folder? > + folderFound = False # Second flag to break inner for > + # Loop only through packages which contain folders > + for pypkg_with_folder in hasfolders: > + if (folderFound == False): > + # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) > + for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: > + if folder_dep == folder: > + print ('%s folder found in %s' % (folder, pypkg_with_folder)) > + folderFound = True > + if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: > + new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) > + else: > + break > + > + # A folder was found so we're done with this item, we can go on > + if inFolders: > + continue > + > + > + > + # No directories beyond this point > + # We might already have this module on the dictionary since it could depend on a (previously checked) module > + if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: > + # Handle core as a special package, we already did it so we pass it to NEW data structure directly > + if pypkg == 'core': > + print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) > + if pymodule_dep.endswith('*'): > + wildcards.append(pymodule_dep) > + if isCached(pymodule_dep): > + new_manifest[pypkg]['cached'].append(pymodule_dep) > + else: > + new_manifest[pypkg]['files'].append(pymodule_dep) > + > + # Check for repeated files > + if pymodule_dep not in allfiles: > + allfiles.append(pymodule_dep) > + else: > + if pymodule_dep not in repeated: > + repeated.append(pymodule_dep) > + else: > + > + > + # Last step: Figure out if we this belongs to FILES or RDEPENDS > + # We check if this module is already contained on another package, so we add that one > + # as an RDEPENDS, or if its not, it means it should be contained on the current > + # package, and we should add it to FILES > + for possible_rdep in old_manifest: > + # Debug > + # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) > + if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: > + # Since were nesting, we need to check its not the same pypkg > + if(possible_rdep != pypkg): > + if possible_rdep not in new_manifest[pypkg]['rdepends']: > + # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs > + reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) > + new_manifest[pypkg]['rdepends'].append(possible_rdep) > + break > + else: > + > + # Since this module wasnt found on another package, it is not an RDEP, > + # so we add it to FILES for this package. > + # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) > + if os.path.basename(pymodule_dep) != pypkg: > + reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) > + if isCached(pymodule_dep): > + new_manifest[pypkg]['cached'].append(pymodule_dep) > + else: > + new_manifest[pypkg]['files'].append(pymodule_dep) > + if pymodule_dep.endswith('*'): > + wildcards.append(pymodule_dep) > + if pymodule_dep not in allfiles: > + allfiles.append(pymodule_dep) > + else: > + if pymodule_dep not in repeated: > + repeated.append(pymodule_dep) > + > + print('\n') > + print('#################################') > + print('Summary for module %s' % pymodule) > + print('FILES found for module %s:' % pymodule) > + print(''.join(reportFILES)) > + print('RDEPENDS found for module %s:' % pymodule) > + print(''.join(reportRDEPS)) > + print('#################################') > + > +print('The following FILES contain wildcards, please check if they are necessary') > +print(wildcards) > +print('The following FILES contain folders, please check if they are necessary') > +print(hasfolders) > + > + > +# Sort it just so it looks nicer > +for pypkg in new_manifest: > + new_manifest[pypkg]['files'].sort() > + new_manifest[pypkg]['cached'].sort() > + new_manifest[pypkg]['rdepends'].sort() > + > +# Create the manifest from the data structure that was built > +with open('python3-manifest.json.new','w') as outfile: > + json.dump(new_manifest,outfile, indent=4) > + outfile.write('\n') > + > +prepend_comments(comments,'python3-manifest.json.new') > + > +if (repeated): > + error_msg = '\n\nERROR:\n' > + error_msg += 'The following files are repeated (contained in more than one package),\n' > + error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' > + error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' > + error_msg += '\n'.join(repeated) > + error_msg += '\n' > + sys.exit(error_msg) > + > diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > new file mode 100644 > index 00000000000..fd12baad84e > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > @@ -0,0 +1,146 @@ > +# This script is launched on separate task for each python module > +# It checks for dependencies for that specific module and prints > +# them out, the output of this execution will have all dependencies > +# for a specific module, which will be parsed an dealt on create_manifest.py > +# > +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com> > + > +# We can get a log per module, for all the dependencies that were found, but its messy. > +debug=False > + > +import sys > + > +# We can get a list of the modules which are currently required to run python > +# so we run python-core and get its modules, we then import what we need > +# and check what modules are currently running, if we substract them from the > +# modules we had initially, we get the dependencies for the module we imported. > + > +# We use importlib to achieve this, so we also need to know what modules importlib needs > +import importlib > + > +core_deps=set(sys.modules) > + > +def fix_path(dep_path): > + import os > + # We DONT want the path on our HOST system > + pivot='recipe-sysroot-native' > + dep_path=dep_path[dep_path.find(pivot)+len(pivot):] > + > + if '/usr/bin' in dep_path: > + dep_path = dep_path.replace('/usr/bin''${bindir}') > + > + # Handle multilib, is there a better way? > + if '/usr/lib32' in dep_path: > + dep_path = dep_path.replace('/usr/lib32','${libdir}') > + if '/usr/lib64' in dep_path: > + dep_path = dep_path.replace('/usr/lib64','${libdir}') > + if '/usr/lib' in dep_path: > + dep_path = dep_path.replace('/usr/lib','${libdir}') > + if '/usr/include' in dep_path: > + dep_path = dep_path.replace('/usr/include','${includedir}') > + if '__init__.' in dep_path: > + dep_path = os.path.split(dep_path)[0] > + return dep_path > + > + > +# Module to import was passed as an argument > +current_module = str(sys.argv[1]).rstrip() > +if(debug==True): > + log = open('log_%s' % current_module,'w') > + log.write('Module %s generated the following dependencies:\n' % current_module) > +try: > + importlib.import_module('%s' % current_module) > +except ImportError as e: > + if (debug==True): > + log.write('Module was not found') > + pass > + > + > +# Get current module dependencies, dif will contain a list of specific deps for this module > +module_deps=set(sys.modules) > + > +# We handle the core package (1st pass on create_manifest.py) as a special case > +if current_module == 'python-core-package': > + dif = core_deps > +else: > + # We know this is not the core package, so there must be a difference. > + dif = module_deps-core_deps > + > + > +# Check where each dependency came from > +for item in dif: > + dep_path='' > + try: > + if (debug==True): > + log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') > + dep_path = sys.modules['%s' % item].__file__ > + except AttributeError as e: > + # Deals with thread (builtin module) not having __file__ attribute > + if debug==True: > + log.write(item + ' ') > + log.write(str(e)) > + log.write('\n') > + pass > + except NameError as e: > + # Deals with NameError: name 'dep_path' is not defined > + # because module is not found (wasn't compiled?), e.g. bddsm > + if (debug==True): > + log.write(item+' ') > + log.write(str(e)) > + pass > + > + # Site-customize is a special case since we (OpenEmbedded) put it there manually > + if 'sitecustomize' in dep_path: > + dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' > + # Prints out result, which is what will be used by create_manifest > + print (dep_path) > + continue > + > + dep_path = fix_path(dep_path) > + > + import sysconfig > + soabi=sysconfig.get_config_var('SOABI') > + # Check if its a shared library and deconstruct it > + if soabi in dep_path: > + if (debug==True): > + log.write('Shared library found in %s' % dep_path) > + dep_path = dep_path.replace(soabi,'*') > + print (dep_path) > + continue > + > + if (debug==True): > + log.write(dep_path+'\n') > + # Prints out result, which is what will be used by create_manifest > + print (dep_path) > + > + > + import imp > + cpython_tag = imp.get_tag() > + cached='' > + # Theres no naive way to find *.pyc files on python3 > + try: > + if (debug==True): > + log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') > + cached = sys.modules['%s' % item].__cached__ > + except AttributeError as e: > + # Deals with thread (builtin module) not having __cached__ attribute > + if debug==True: > + log.write(item + ' ') > + log.write(str(e)) > + log.write('\n') > + pass > + except NameError as e: > + # Deals with NameError: name 'cached' is not defined > + if (debug==True): > + log.write(item+' ') > + log.write(str(e)) > + pass > + if cached is not None: > + if (debug==True): > + log.write(cached) > + cached = fix_path(cached) > + cached = cached.replace(cpython_tag,'*') > + print (cached) > + > +if debug==True: > + log.close() > diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch > new file mode 100644 > index 00000000000..f23b8b7df06 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch > @@ -0,0 +1,46 @@ > +python-config: Revert to using distutils.sysconfig > + > +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in > + > +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig > + > +affect the native runtime as well as cross building. Use the old, patched > +implementation which returns paths in the staging directory and for the target, > +as appropriate. > + > +Upstream-Status: Inappropriate [Embedded Specific] > + > +Signed-off-by: Tyler Hall <tylerwhall@gmail.com> > +: > +Index: Python-3.3.3/Misc/python-config.in > +=================================================================== > +--- Python-3.3.3.orig/Misc/python-config.in > ++++ Python-3.3.3/Misc/python-config.in > +@@ -4,7 +4,7 @@ > + import getopt > + import os > + import sys > +-import sysconfig > ++from distutils import sysconfig > + > + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', > + 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] > +@@ -32,14 +32,14 @@ if '--help' in opt_flags: > + > + for opt in opt_flags: > + if opt == '--prefix': > +- print(sysconfig.get_config_var('prefix')) > ++ print(sysconfig.PREFIX) > + > + elif opt == '--exec-prefix': > +- print(sysconfig.get_config_var('exec_prefix')) > ++ print(sysconfig.EXEC_PREFIX) > + > + elif opt in ('--includes', '--cflags'): > +- flags = ['-I' + sysconfig.get_path('include'), > +- '-I' + sysconfig.get_path('platinclude')] > ++ flags = ['-I' + sysconfig.get_python_inc(), > ++ '-I' + sysconfig.get_python_inc(plat_specific=True)] > + if opt == '--cflags': > + flags.extend(getvar('CFLAGS').split()) > + print(' '.join(flags)) > diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json > new file mode 100644 > index 00000000000..82c7075f0e7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json > @@ -0,0 +1,1227 @@ > +# DO NOT (entirely) modify this file manually, please read. > +# > +# IMPORTANT NOTE: > +# Please keep in mind that the create_manifest task relies on the fact the the > +# target and native Python packages are the same, and it also needs to be executed > +# with a fully working native package (with all the PACKAGECONFIGs enabled and all > +# and all the modules should be working, check log.do_compile), otherwise the script > +# will fail to find dependencies correctly, this note is valid either if you are > +# upgrading to a new Python version or adding a new package. > +# > +# > +# If you are adding a new package please follow the next steps: > +# How to add a new package: > +# - If a user wants to add a new package all that has to be done is: > +# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, > +# fill up the SUMMARY section as well, the script should handle all the rest. > +# > +# Real example: > +# We want to add a web browser package, including the file webbrowser.py > +# which at the moment is on python3-misc. > +# "webbrowser": { > +# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], > +# "rdepends": [], > +# "summary": "Python Web Browser support"} > +# > +# * Note that the rdepends field was left empty > +# > +# We run $ bitbake python3 -c create_manifest and the resulting manifest > +# should be completed after a few seconds, showing something like: > +# "webbrowser": { > +# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], > +# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], > +# "summary": "Python Web Browser support"} > +# > +# > +# If you are upgrading Python to a new version please follow the next steps: > +# After each Python upgrade, the create_manifest task should be executed, because we > +# don't control what changes on upstream Python, so, some module dependency > +# might have changed without us realizing it, a certain module can either have > +# more or less dependencies, or could be depending on a new file that was just > +# created on the new release and for obvious reasons we wouldn't have it on our > +# old manifest, all of these issues would cause runtime errors on our system. > +# > +# - Upgrade both the native and target Python packages to a new version > +# - Run the create_manifest task for the target Python package as its shown below: > +# > +# $ bitbake python3 -c create_manifest > +# > +# This will automatically replace your manifest file located under the Python directory > +# with an new one, which contains the new dependencies (if any). > +# > +# Several things could have gone wrong here, I will try to explain a few: > +# > +# a) A new file was introduced on this release, e.g. sha3*.so: > +# The task will check what its needed to import every module, more than one module would > +# would probably depend on sha3*.so, although only one module should contain it. > +# > +# After running the task, the new manifest will have the sha3*.so file on more than one > +# module, you need to manually decide which one of them should get it and delete it from > +# the others, for example sha3*.so should likely be on ${PN}-crypt. > +# Once you have deleted from the others you need to run the create_manifest task again, > +# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be > +# good to go. > +# > +# b) The native package wasn't built correctly and its missing a certain module: > +# As mentioned before, you need to make sure the native package was built with all the modules > +# because it is used as base to build the manifest file, you need to manually check log.do_compile > +# since it won't error out the compile function if its only missing a couple of modules. > +# > +# e.g. missing the _uuid module, log.do_compile would show the following: > +# Python build finished successfully! > +# The necessary bits to build these optional modules were not found: > +# _uuid > +# > +# What will happen here is that the new manifest would not be aware that the _uuid module exists, so > +# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on > +# the misc package (which is where any file that doesn't belong anywhere else ends up). > +# > +# This will eventually cause runtime errors on our system if we don't include the misc package on > +# on our image, because the _uuid files will be missing. > +# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be > +# detected correctly along with its dependencies, and we will get a working manifest. > +# > +# This is the reason why it is important to make sure we have a fully working native build, > +# so we can avoid these errors. > +# > +# > +# > +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST > +# EOC > +{ > + "tests": { > + "summary": "Python test suite", > + "rdepends": [ > + "core", > + "modules" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/*/test", > + "${libdir}/python${PYTHON_MAJMIN}/*/tests", > + "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", > + "${libdir}/python${PYTHON_MAJMIN}/test" > + ], > + "cached": [] > + }, > + "2to3": { > + "summary": "Python automated Python 2 to 3 code translator", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${bindir}/2to3-*", > + "${libdir}/python${PYTHON_MAJMIN}/lib2to3" > + ], > + "cached": [] > + }, > + "asyncio": { > + "summary": "Python Asynchronous I/", > + "rdepends": [ > + "core", > + "io", > + "logging", > + "netclient", > + "numbers", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/asyncio", > + "${libdir}/python${PYTHON_MAJMIN}/concurrent", > + "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so" > + ], > + "cached": [] > + }, > + "audio": { > + "summary": "Python Audio Handling", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/chunk.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", > + "${libdir}/python${PYTHON_MAJMIN}/sunau.py", > + "${libdir}/python${PYTHON_MAJMIN}/wave.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" > + ] > + }, > + "codecs": { > + "summary": "Python codec", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" > + ] > + }, > + "compile": { > + "summary": "Python bytecode compilation support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/compileall.py", > + "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" > + ] > + }, > + "compression": { > + "summary": "Python high-level compression support", > + "rdepends": [ > + "core", > + "shell", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_compression.py", > + "${libdir}/python${PYTHON_MAJMIN}/bz2.py", > + "${libdir}/python${PYTHON_MAJMIN}/gzip.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lzma.py", > + "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", > + "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" > + ] > + }, > + "core": { > + "summary": "Python interpreter and core modules", > + "rdepends": [], > + "files": [ > + "${bindir}/python*[!-config]", > + "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", > + "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", > + "${libdir}/python${PYTHON_MAJMIN}/UserList.py", > + "${libdir}/python${PYTHON_MAJMIN}/UserString.py", > + "${libdir}/python${PYTHON_MAJMIN}/__future__.py", > + "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", > + "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", > + "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", > + "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", > + "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", > + "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", > + "${libdir}/python${PYTHON_MAJMIN}/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/argparse.py", > + "${libdir}/python${PYTHON_MAJMIN}/ast.py", > + "${libdir}/python${PYTHON_MAJMIN}/bisect.py", > + "${libdir}/python${PYTHON_MAJMIN}/code.py", > + "${libdir}/python${PYTHON_MAJMIN}/codecs.py", > + "${libdir}/python${PYTHON_MAJMIN}/codeop.py", > + "${libdir}/python${PYTHON_MAJMIN}/collections", > + "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/configparser.py", > + "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/copy.py", > + "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", > + "${libdir}/python${PYTHON_MAJMIN}/csv.py", > + "${libdir}/python${PYTHON_MAJMIN}/dis.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", > + "${libdir}/python${PYTHON_MAJMIN}/enum.py", > + "${libdir}/python${PYTHON_MAJMIN}/functools.py", > + "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", > + "${libdir}/python${PYTHON_MAJMIN}/getopt.py", > + "${libdir}/python${PYTHON_MAJMIN}/gettext.py", > + "${libdir}/python${PYTHON_MAJMIN}/heapq.py", > + "${libdir}/python${PYTHON_MAJMIN}/imp.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", > + "${libdir}/python${PYTHON_MAJMIN}/inspect.py", > + "${libdir}/python${PYTHON_MAJMIN}/io.py", > + "${libdir}/python${PYTHON_MAJMIN}/keyword.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/linecache.py", > + "${libdir}/python${PYTHON_MAJMIN}/locale.py", > + "${libdir}/python${PYTHON_MAJMIN}/new.py", > + "${libdir}/python${PYTHON_MAJMIN}/opcode.py", > + "${libdir}/python${PYTHON_MAJMIN}/operator.py", > + "${libdir}/python${PYTHON_MAJMIN}/optparse.py", > + "${libdir}/python${PYTHON_MAJMIN}/os.py", > + "${libdir}/python${PYTHON_MAJMIN}/platform.py", > + "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", > + "${libdir}/python${PYTHON_MAJMIN}/re.py", > + "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", > + "${libdir}/python${PYTHON_MAJMIN}/selectors.py", > + "${libdir}/python${PYTHON_MAJMIN}/signal.py", > + "${libdir}/python${PYTHON_MAJMIN}/site.py", > + "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", > + "${libdir}/python${PYTHON_MAJMIN}/stat.py", > + "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", > + "${libdir}/python${PYTHON_MAJMIN}/struct.py", > + "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", > + "${libdir}/python${PYTHON_MAJMIN}/symbol.py", > + "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", > + "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", > + "${libdir}/python${PYTHON_MAJMIN}/threading.py", > + "${libdir}/python${PYTHON_MAJMIN}/token.py", > + "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", > + "${libdir}/python${PYTHON_MAJMIN}/traceback.py", > + "${libdir}/python${PYTHON_MAJMIN}/types.py", > + "${libdir}/python${PYTHON_MAJMIN}/warnings.py", > + "${libdir}/python${PYTHON_MAJMIN}/weakref.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" > + ] > + }, > + "crypt": { > + "summary": "Python basic cryptographic and hashing support", > + "rdepends": [ > + "core", > + "math", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/crypt.py", > + "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" > + ] > + }, > + "ctypes": { > + "summary": "Python C types support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/ctypes", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" > + ], > + "cached": [] > + }, > + "curses": { > + "summary": "Python curses support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/curses", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" > + ], > + "cached": [] > + }, > + "datetime": { > + "summary": "Python calendar and time support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", > + "${libdir}/python${PYTHON_MAJMIN}/calendar.py", > + "${libdir}/python${PYTHON_MAJMIN}/datetime.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" > + ] > + }, > + "db": { > + "summary": "Python file-based database support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/dbm", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" > + ], > + "cached": [] > + }, > + "debugger": { > + "summary": "Python debugger", > + "rdepends": [ > + "core", > + "pprint", > + "shell", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/bdb.py", > + "${libdir}/python${PYTHON_MAJMIN}/pdb.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" > + ] > + }, > + "dev": { > + "cached": [], > + "files": [ > + "${base_libdir}/*.a", > + "${base_libdir}/*.o", > + "${bindir}/python*-config", > + "${datadir}/aclocal", > + "${datadir}/pkgconfig", > + "${includedir}", > + "${libdir}/*.a", > + "${libdir}/*.la", > + "${libdir}/*.o", > + "${libdir}/lib*${SOLIBSDEV}", > + "${libdir}/pkgconfig", > + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", > + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" > + ], > + "rdepends": [ > + "core" > + ], > + "summary": "Python development package" > + }, > + "difflib": { > + "summary": "Python helpers for computing deltas between objects", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/difflib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" > + ] > + }, > + "distutils-staticdev": { > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" > + ], > + "rdepends": [ > + "distutils" > + ], > + "summary": "Python distribution utilities (static libraries)" > + }, > + "distutils": { > + "summary": "Python Distribution Utilities", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/distutils" > + ], > + "cached": [] > + }, > + "doctest": { > + "summary": "Python framework for running examples in docstrings", > + "rdepends": [ > + "core", > + "debugger", > + "difflib", > + "logging", > + "pprint", > + "shell", > + "stringold", > + "unittest" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/doctest.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" > + ] > + }, > + "email": { > + "summary": "Python email support", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "io", > + "math", > + "netclient" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/email", > + "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" > + ] > + }, > + "fcntl": { > + "summary": "Python's fcntl interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" > + ], > + "cached": [] > + }, > + "gdbm": { > + "summary": "Python GNU database support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" > + ], > + "cached": [] > + }, > + "html": { > + "summary": "Python HTML processing support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/formatter.py", > + "${libdir}/python${PYTHON_MAJMIN}/html" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" > + ] > + }, > + "idle": { > + "summary": "Python Integrated Development Environment", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${bindir}/idle*", > + "${libdir}/python${PYTHON_MAJMIN}/idlelib" > + ], > + "cached": [] > + }, > + "image": { > + "summary": "Python graphical image handling", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", > + "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" > + ] > + }, > + "io": { > + "summary": "Python low-level I/O", > + "rdepends": [ > + "compression", > + "core", > + "crypt", > + "math", > + "netclient", > + "shell", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", > + "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/pipes.py", > + "${libdir}/python${PYTHON_MAJMIN}/socket.py", > + "${libdir}/python${PYTHON_MAJMIN}/ssl.py", > + "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" > + ] > + }, > + "json": { > + "summary": "Python JSON support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/json", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" > + ], > + "cached": [] > + }, > + "logging": { > + "summary": "Python logging support", > + "rdepends": [ > + "core", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/logging" > + ], > + "cached": [] > + }, > + "mailbox": { > + "summary": "Python mailbox format support", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "fcntl", > + "io", > + "math", > + "mime", > + "netclient", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" > + ] > + }, > + "math": { > + "summary": "Python math support", > + "rdepends": [ > + "core", > + "crypt" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/random.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" > + ] > + }, > + "mime": { > + "summary": "Python MIME handling APIs", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/quopri.py", > + "${libdir}/python${PYTHON_MAJMIN}/uu.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" > + ] > + }, > + "mmap": { > + "summary": "Python memory-mapped file support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" > + ], > + "cached": [] > + }, > + "modules": { > + "cached": [], > + "files": [], > + "rdepends": [ > + "2to3", > + "asyncio", > + "audio", > + "codecs", > + "compile", > + "compression", > + "core", > + "crypt", > + "ctypes", > + "curses", > + "datetime", > + "db", > + "debugger", > + "difflib", > + "distutils", > + "doctest", > + "email", > + "fcntl", > + "html", > + "idle", > + "image", > + "io", > + "json", > + "logging", > + "mailbox", > + "math", > + "mime", > + "mmap", > + "multiprocessing", > + "netclient", > + "netserver", > + "numbers", > + "pickle", > + "pkgutil", > + "plistlib", > + "pprint", > + "profile", > + "pydoc", > + "resource", > + "runpy", > + "shell", > + "smtpd", > + "sqlite3", > + "stringold", > + "syslog", > + "terminal", > + "threading", > + "tkinter", > + "typing", > + "unittest", > + "unixadmin", > + "venv", > + "xml", > + "xmlrpc" > + ], > + "summary": "All Python modules" > + }, > + "multiprocessing": { > + "summary": "Python multiprocessing support", > + "rdepends": [ > + "core", > + "io", > + "pickle" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" > + ], > + "cached": [] > + }, > + "netclient": { > + "summary": "Python Internet Protocol clients", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "io", > + "math", > + "mime", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/base64.py", > + "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/hmac.py", > + "${libdir}/python${PYTHON_MAJMIN}/http", > + "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", > + "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/poplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/urllib", > + "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/uuid.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" > + ] > + }, > + "netserver": { > + "summary": "Python Internet Protocol servers", > + "rdepends": [ > + "compression", > + "core", > + "crypt", > + "datetime", > + "email", > + "html", > + "io", > + "math", > + "mime", > + "netclient", > + "shell", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cgi.py", > + "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" > + ] > + }, > + "numbers": { > + "summary": "Python number APIs", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", > + "${libdir}/python${PYTHON_MAJMIN}/contextvars.py", > + "${libdir}/python${PYTHON_MAJMIN}/decimal.py", > + "${libdir}/python${PYTHON_MAJMIN}/fractions.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/numbers.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" > + ] > + }, > + "pickle": { > + "summary": "Python serialisation/persistence support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/pickle.py", > + "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", > + "${libdir}/python${PYTHON_MAJMIN}/shelve.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" > + ] > + }, > + "pkgutil": { > + "summary": "Python package extension utility support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" > + ] > + }, > + "plistlib": { > + "summary": "Generate and parse Mac OS X .plist files", > + "rdepends": [ > + "core", > + "datetime", > + "xml" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" > + ] > + }, > + "pprint": { > + "summary": "Python pretty-print support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pprint.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" > + ] > + }, > + "profile": { > + "summary": "Python basic performance profiling support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/profile.py", > + "${libdir}/python${PYTHON_MAJMIN}/pstats.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" > + ] > + }, > + "pydoc": { > + "summary": "Python interactive help support", > + "rdepends": [ > + "core", > + "netclient", > + "pkgutil" > + ], > + "files": [ > + "${bindir}/pydoc*", > + "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", > + "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" > + ] > + }, > + "resource": { > + "summary": "Python resource control interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" > + ], > + "cached": [] > + }, > + "runpy": { > + "summary": "Python helper for locating/executing scripts in module namespace", > + "rdepends": [ > + "core", > + "pkgutil" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/runpy.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" > + ] > + }, > + "shell": { > + "summary": "Python shell-like functionality", > + "rdepends": [ > + "compression", > + "core", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cmd.py", > + "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", > + "${libdir}/python${PYTHON_MAJMIN}/glob.py", > + "${libdir}/python${PYTHON_MAJMIN}/shlex.py", > + "${libdir}/python${PYTHON_MAJMIN}/shutil.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" > + ] > + }, > + "smtpd": { > + "summary": "Python Simple Mail Transport Daemon", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "io", > + "math", > + "mime", > + "netclient", > + "stringold" > + ], > + "files": [ > + "${bindir}/smtpd.py", > + "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", > + "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", > + "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" > + ] > + }, > + "sqlite3": { > + "summary": "Python Sqlite3 database support", > + "rdepends": [ > + "core", > + "datetime" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/sqlite3" > + ], > + "cached": [] > + }, > + "stringold": { > + "summary": "Python string APIs [deprecated]", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/string.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" > + ] > + }, > + "syslog": { > + "summary": "Python syslog interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" > + ], > + "cached": [] > + }, > + "terminal": { > + "summary": "Python terminal controlling support", > + "rdepends": [ > + "core", > + "io" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pty.py", > + "${libdir}/python${PYTHON_MAJMIN}/tty.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" > + ] > + }, > + "threading": { > + "summary": "Python threading & synchronization support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", > + "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/queue.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" > + ] > + }, > + "tkinter": { > + "summary": "Python Tcl/Tk bindings", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/tkinter" > + ], > + "cached": [] > + }, > + "typing": { > + "summary": "Python typing support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/typing.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" > + ] > + }, > + "unittest": { > + "summary": "Python unit testing framework", > + "rdepends": [ > + "core", > + "difflib", > + "logging", > + "pprint", > + "shell", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/unittest", > + "${libdir}/python${PYTHON_MAJMIN}/unittest/", > + "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" > + ], > + "cached": [] > + }, > + "unixadmin": { > + "summary": "Python Unix administration support", > + "rdepends": [ > + "core", > + "io" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/getpass.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" > + ] > + }, > + "venv": { > + "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", > + "rdepends": [ > + "compression", > + "core", > + "logging", > + "shell", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${bindir}/pyvenv*", > + "${libdir}/python${PYTHON_MAJMIN}/venv" > + ], > + "cached": [] > + }, > + "xml": { > + "summary": "Python basic XML support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/xml" > + ], > + "cached": [] > + }, > + "xmlrpc": { > + "summary": "Python XML-RPC support", > + "rdepends": [ > + "core", > + "xml" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", > + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" > + ], > + "cached": [] > + } > +} > diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest > new file mode 100644 > index 00000000000..3863c6d314f > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest > @@ -0,0 +1,3 @@ > +#!/bin/sh > + > +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' > diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb > new file mode 100644 > index 00000000000..61fa12ffe8d > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb > @@ -0,0 +1,281 @@ > +SUMMARY = "The Python Programming Language" > +HOMEPAGE = "http://www.python.org" > +LICENSE = "PSFv2" > +SECTION = "devel/python" > + > +LIC_FILES_CHKSUM = "file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754" > + > +SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ > + file://run-ptest \ > + file://create_manifest3.py \ > + file://get_module_deps3.py \ > + file://python3-manifest.json \ > + file://check_build_completeness.py \ > + file://cgi_py.patch \ > + file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \ > + ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \ > + file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \ > + file://python-config.patch \ > + file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \ > + " > + > +SRC_URI_append_class-native = " \ > + file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ > + file://12-distutils-prefix-is-inside-staging-area.patch \ > + " > + > +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb" > +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb" > + > +# exclude pre-releases for both python 2.x and 3.x > +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" > + > +CVE_PRODUCT = "python" > + > +PYTHON_MAJMIN = "3.7" > +PYTHON_BINABI = "${PYTHON_MAJMIN}m" > + > +S = "${WORKDIR}/Python-${PV}" > + > +BBCLASSEXTEND = "native nativesdk" > + > +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives > + > +MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}" > + > +ALTERNATIVE_${PN}-dev = "python-config" > +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" > +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" > + > + > +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2" > +DEPENDS_append_class-target = " python3-native" > +DEPENDS_append_class-nativesdk = " python3-native" > + > +EXTRA_OECONF = " --without-ensurepip --enable-shared" > +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}" > + > + > +EXTRANATIVEPATH += "python3-native" > + > +CACHED_CONFIGUREVARS = " \ > + ac_cv_file__dev_ptmx=yes \ > + ac_cv_file__dev_ptc=no \ > +" > + > +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" > +PACKAGECONFIG_class-native ??= "readline" > +PACKAGECONFIG_class-nativesdk ??= "readline" > +PACKAGECONFIG[readline] = ",,readline" > +# Use profile guided optimisation by running PyBench inside qemu-user > +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native" > +PACKAGECONFIG[tk] = ",,tk" > + > +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid" > + > +EXTRA_OEMAKE = '\ > + STAGING_LIBDIR=${STAGING_LIBDIR} \ > + STAGING_INCDIR=${STAGING_INCDIR} \ > +' > + > +do_compile_prepend_class-target() { > + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then > + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" > + cat >pgo-wrapper <<EOF > +#!/bin/sh > +cd ${B} > +$qemu_binary "\$@" > +EOF > + chmod +x pgo-wrapper > + fi > +} > + > +do_install_prepend() { > + ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile > +} > + > +do_install_append_class-target() { > + oe_multilib_header python${PYTHON_BINABI}/pyconfig.h > +} > + > +do_install_append_class-native() { > + # Make sure we use /usr/bin/env python > + for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do > + sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT > + done > + # Add a symlink to the native Python so that scripts can just invoke > + # "nativepython" and get the right one without needing absolute paths > + # (these often end up too long for the #! parser in the kernel as the > + # buffer is 128 bytes long). > + ln -s python3-native/python3 ${D}${bindir}/nativepython3 > +} > + > +do_install_append() { > + mkdir -p ${D}${libdir}/python-sysconfigdata > + sysconfigfile=`find ${D} -name _sysconfig*.py` > + cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py > + > + sed -i \ > + -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \ > + -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ > + -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ > + ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py > +} > + > +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py" > +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" > + > +py_package_preprocess () { > + # Remove references to buildmachine paths in target Makefile and _sysconfigdata > + sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ > + -e 's|${DEBUG_PREFIX_MAP}||g' \ > + -e 's:${HOSTTOOLS_DIR}/::g' \ > + -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ > + -e 's:${RECIPE_SYSROOT}::g' \ > + -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ > + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \ > + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \ > + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config > + > + # Recompile _sysconfigdata after modifying it > + cd ${PKGD} > + sysconfigfile=`find . -name _sysconfigdata_*.py` > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile')" > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)" > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)" > + cd - > + > + mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} > + > + #Remove the unneeded copy of target sysconfig data > + rm -rf ${PKGD}/${libdir}/python-sysconfigdata > +} > + > +# We want bytecode precompiled .py files (.pyc's) by default > +# but the user may set it on their own conf > +INCLUDE_PYCS ?= "1" > + > +python(){ > + import collections, json > + > + filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') > + # This python changes the datastore based on the contents of a file, so mark > + # that dependency. > + bb.parse.mark_dependency(d, filename) > + > + with open(filename) as manifest_file: > + manifest_str = manifest_file.read() > + json_start = manifest_str.find('# EOC') + 6 > + manifest_file.seek(json_start) > + manifest_str = manifest_file.read() > + python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) > + > + # First set RPROVIDES for -native case > + # Hardcoded since it cant be python3-native-foo, should be python3-foo-native > + pn = 'python3' > + rprovides = d.getVar('RPROVIDES').split() > + > + for key in python_manifest: > + pypackage = pn + '-' + key + '-native' > + if pypackage not in rprovides: > + rprovides.append(pypackage) > + > + d.setVar('RPROVIDES_class-native', ' '.join(rprovides)) > + > + # Then work on the target > + include_pycs = d.getVar('INCLUDE_PYCS') > + > + packages = d.getVar('PACKAGES').split() > + pn = d.getVar('PN') > + > + newpackages=[] > + for key in python_manifest: > + pypackage= pn + '-' + key > + > + if pypackage not in packages: > + # We need to prepend, otherwise python-misc gets everything > + # so we use a new variable > + newpackages.append(pypackage) > + > + # "Build" python's manifest FILES, RDEPENDS and SUMMARY > + d.setVar('FILES_' + pypackage, '') > + for value in python_manifest[key]['files']: > + d.appendVar('FILES_' + pypackage, ' ' + value) > + > + # Add cached files > + if include_pycs == '1': > + for value in python_manifest[key]['cached']: > + d.appendVar('FILES_' + pypackage, ' ' + value) > + > + for value in python_manifest[key]['rdepends']: > + # Make it work with or without $PN > + if '${PN}' in value: > + value=value.split('-')[1] > + d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) > + d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) > + > + # Prepending so to avoid python-misc getting everything > + packages = newpackages + packages > + d.setVar('PACKAGES', ' '.join(packages)) > + d.setVar('ALLOW_EMPTY_${PN}-modules', '1') > +} > + > +# Files needed to create a new manifest > + > +do_create_manifest() { > + # This task should be run with every new release of Python. > + # We must ensure that PACKAGECONFIG enables everything when creating > + # a new manifest, this is to base our new manifest on a complete > + # native python build, containing all dependencies, otherwise the task > + # wont be able to find the required files. > + # e.g. BerkeleyDB is an optional build dependency so it may or may not > + # be present, we must ensure it is. > + > + cd ${WORKDIR} > + # This needs to be executed by python-native and NOT by HOST's python > + nativepython3 create_manifest3.py ${PYTHON_MAJMIN} > + cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json > +} > + > +# bitbake python -c create_manifest > +addtask do_create_manifest > + > +# Make sure we have native python ready when we create a new manifest > +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" > +do_create_manifest[depends] += "python3:do_patch" > + > +# manual dependency additions > +RPROVIDES_${PN}-modules = "${PN}" > +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" > +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" > + > +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" > +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" > +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" > + > +# provide python-pyvenv from python3-venv > +RPROVIDES_${PN}-venv += "python3-pyvenv" > + > +# package libpython3 > +PACKAGES =+ "libpython3 libpython3-staticdev" > +FILES_libpython3 = "${libdir}/libpython*.so.*" > +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a" > +INSANE_SKIP_${PN}-dev += "dev-elf" > + > +# catch all the rest (unsorted) > +PACKAGES += "${PN}-misc" > +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs" > +RDEPENDS_${PN}-modules += "python3-misc" > +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" > + > +# catch manpage > +PACKAGES += "${PN}-man" > +FILES_${PN}-man = "${datadir}/man" > + > +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" > +RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" > +RDEPENDS_${PN}-dev = "" > + > -- > 2.17.1 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] python3: upgrade to 3.7.2 2019-02-01 20:49 ` [PATCH 1/3] python3: upgrade to 3.7.2 Khem Raj @ 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego 2019-02-01 22:35 ` Alejandro Enedino Hernandez Samaniego ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Alejandro Enedino Hernandez Samaniego @ 2019-02-01 22:32 UTC (permalink / raw) To: Khem Raj, Alexander Kanavin Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 118126 bytes --] On 2/1/19 12:49 PM, Khem Raj wrote: > On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: >> I took the same approach as the recent perl upgrade: write recipe from scratch, >> taking the pieces from the old recipe only when they were proven to be necessary. >> >> The pgo, manifest and ptest features are all preserved. >> >> New features: >> >> - native and target recipes are now unified into one recipe >> >> - check_build_completeness.py runs right after do_compile() and verifies that >> all optional modules have been built (a notorious source of regressions) >> >> - a new approach to sysconfig.py and distutils/sysconfig.py returning values >> appropriate for native or target builds: we copy the configuration file to a >> separate folder, add that folder to sys.path (through environment variable >> that differs between native and target builds), and point python to the file >> through another environment variable. >> > Thanks, once current cycle is over for OE, I would like to give this a > whirl and see how it goes > >> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> >> --- >> meta/classes/python3-dir.bbclass | 2 +- >> meta/classes/python3native.bbclass | 2 + >> ...ib-termcap-to-linker-flags-to-avoid-.patch | 25 + >> ...hell-version-of-python-config-that-w.patch | 35 + >> ...-qemu-wrapper-when-gathering-profile.patch | 25 + >> ...fig-append-STAGING_LIBDIR-python-sys.patch | 42 + >> ...tutils-prefix-is-inside-staging-area.patch | 54 + >> .../python3/avoid_warning_about_tkinter.patch | 36 + >> .../python-sanity/python3/cgi_py.patch | 32 + >> .../python3/check_build_completeness.py | 17 + >> .../python-sanity/python3/create_manifest3.py | 433 ++++++ >> .../python-sanity/python3/get_module_deps3.py | 146 ++ >> .../python-sanity/python3/python-config.patch | 46 + >> .../python3/python3-manifest.json | 1227 +++++++++++++++++ >> .../python-sanity/python3/run-ptest | 3 + >> .../python-sanity/python3_3.7.2.bb | 281 ++++ >> 16 files changed, 2405 insertions(+), 1 deletion(-) >> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch >> create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >> create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py >> create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >> create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch >> create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json >> create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest >> create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb Thanks Alex!, I looked at this briefly and it looks good. Do we want to keep the recipe at recipes-devtools-python-sanity?, also I think the PACKAGECONFIG for tk is broken but thats probably tk's fault, and, since tk is part of meta-oe only, I am wondering if it would make sense to have a bbappend there with the PACKAGECONFIG there, since we're depending on something that were actually not holding? Alejandro >> >> diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass >> index 06bb046d9c2..7dd130bad99 100644 >> --- a/meta/classes/python3-dir.bbclass >> +++ b/meta/classes/python3-dir.bbclass >> @@ -1,4 +1,4 @@ >> -PYTHON_BASEVERSION = "3.5" >> +PYTHON_BASEVERSION = "3.7" >> PYTHON_ABI = "m" >> PYTHON_DIR = "python${PYTHON_BASEVERSION}" >> PYTHON_PN = "python3" >> diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass >> index da12a714703..a3acaf61bbd 100644 >> --- a/meta/classes/python3native.bbclass >> +++ b/meta/classes/python3native.bbclass >> @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native " >> export STAGING_INCDIR >> export STAGING_LIBDIR >> >> +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" >> + >> # suppress host user's site-packages dirs. >> export PYTHONNOUSERSITE = "1" >> >> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >> new file mode 100644 >> index 00000000000..09f279ba1d7 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >> @@ -0,0 +1,25 @@ >> +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001 >> +From: Alexander Kanavin <alex.kanavin@gmail.com> >> +Date: Fri, 25 Jan 2019 19:04:13 +0100 >> +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host >> + contamination >> + >> +Upstream-Status: Inappropriate [oe-core specific] >> +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> >> + >> +--- >> + setup.py | 1 - >> + 1 file changed, 1 deletion(-) >> + >> +diff --git a/setup.py b/setup.py >> +index b4357e3..fbec00d 100644 >> +--- a/setup.py >> ++++ b/setup.py >> +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext): >> + 'termcap'): >> + readline_libs.append('termcap') >> + exts.append( Extension('readline', ['readline.c'], >> +- library_dirs=['/usr/lib/termcap'], >> + extra_link_args=readline_extra_link_args, >> + libraries=readline_libs) ) >> + else: >> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >> new file mode 100644 >> index 00000000000..83fd52d87f4 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >> @@ -0,0 +1,35 @@ >> +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001 >> +From: Alexander Kanavin <alex.kanavin@gmail.com> >> +Date: Tue, 29 Jan 2019 15:03:01 +0100 >> +Subject: [PATCH] Do not use the shell version of python-config that was >> + introduced in 3.4 >> + >> +Revert instead to the original python version: it has our tweaks and >> +outputs directories correctly. >> + >> +Upstream-Status: Inappropriate [oe-specific] >> +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> >> +--- >> + Makefile.pre.in | 9 +++------ >> + 1 file changed, 3 insertions(+), 6 deletions(-) >> + >> +diff --git a/Makefile.pre.in b/Makefile.pre.in >> +index 2d2e11f..cc19942 100644 >> +--- a/Makefile.pre.in >> ++++ b/Makefile.pre.in >> +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh >> + sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py >> + @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} >> + LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config >> +- @ # On Darwin, always use the python version of the script, the shell >> +- @ # version doesn't use the compiler customizations that are provided >> +- @ # in python (_osx_support.py). >> +- @if test `uname -s` = Darwin; then \ >> +- cp python-config.py python-config; \ >> +- fi >> ++ @ # In OpenEmbedded, always use the python version of the script, the shell >> ++ @ # version is broken in multiple ways, and doesn't return correct directories >> ++ cp python-config.py python-config >> + >> + >> + # Install the include files >> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >> new file mode 100644 >> index 00000000000..fa7735ff93e >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >> @@ -0,0 +1,25 @@ >> +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001 >> +From: Alexander Kanavin <alex.kanavin@gmail.com> >> +Date: Wed, 30 Jan 2019 12:41:04 +0100 >> +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data >> + >> +Upstream-Status: Inappropriate [oe-core specific] >> +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> >> +--- >> + Makefile.pre.in | 3 +-- >> + 1 file changed, 1 insertion(+), 2 deletions(-) >> + >> +diff --git a/Makefile.pre.in b/Makefile.pre.in >> +index a3a02a7..d5503dd 100644 >> +--- a/Makefile.pre.in >> ++++ b/Makefile.pre.in >> +@@ -507,8 +507,7 @@ build_all_generate_profile: >> + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" >> + >> + run_profile_task: >> +- @ # FIXME: can't run for a cross build >> +- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true >> ++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true >> + >> + build_all_merge_profile: >> + $(LLVM_PROF_MERGER) >> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >> new file mode 100644 >> index 00000000000..2c5b76f131c >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >> @@ -0,0 +1,42 @@ >> +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001 >> +From: Alexander Kanavin <alex.kanavin@gmail.com> >> +Date: Thu, 31 Jan 2019 16:46:30 +0100 >> +Subject: [PATCH] distutils/sysconfig: append >> + STAGING_LIBDIR/python-sysconfigdata to sys.path >> + >> +So that target configuration can be used when running native python >> + >> +Upstream-Status: Inappropriate [oe-core specific] >> +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> >> + >> +--- >> + Lib/distutils/sysconfig.py | 2 ++ >> + Lib/sysconfig.py | 2 ++ >> + 2 files changed, 4 insertions(+) >> + >> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >> +index e07a6c8..6b8c129 100644 >> +--- a/Lib/distutils/sysconfig.py >> ++++ b/Lib/distutils/sysconfig.py >> +@@ -421,6 +421,8 @@ def _init_posix(): >> + platform=sys.platform, >> + multiarch=getattr(sys.implementation, '_multiarch', ''), >> + )) >> ++ if 'STAGING_LIBDIR' in os.environ: >> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >> + build_time_vars = _temp.build_time_vars >> + global _config_vars >> +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py >> +index 9ee4d31..e586abd 100644 >> +--- a/Lib/sysconfig.py >> ++++ b/Lib/sysconfig.py >> +@@ -412,6 +412,8 @@ def _init_posix(vars): >> + """Initialize the module as appropriate for POSIX systems.""" >> + # _sysconfigdata is generated at build time, see _generate_posix_vars() >> + name = _get_sysconfigdata_name() >> ++ if 'STAGING_LIBDIR' in os.environ: >> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >> + build_time_vars = _temp.build_time_vars >> + vars.update(build_time_vars) >> diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >> new file mode 100644 >> index 00000000000..abf08f59dd7 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >> @@ -0,0 +1,54 @@ >> +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001 >> +From: Khem Raj <raj.khem@gmail.com> >> +Date: Tue, 14 May 2013 15:00:26 -0700 >> +Subject: [PATCH] python3: Add target and native recipes >> + >> +Upstream-Status: Inappropriate [embedded specific] >> + >> +02/2015 Rebased for Python 3.4.2 >> + >> +# The proper prefix is inside our staging area. >> +# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de> >> +# Signed-off-by: Phil Blundell <philb@gnu.org> >> +# Signed-off-by: Khem Raj <raj.khem@gmail.com> >> +# Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> >> + >> +--- >> + Lib/distutils/sysconfig.py | 10 ++++++++-- >> + 1 file changed, 8 insertions(+), 2 deletions(-) >> + >> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >> +index 6b8c129..3ca7f79 100644 >> +--- a/Lib/distutils/sysconfig.py >> ++++ b/Lib/distutils/sysconfig.py >> +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): >> + If 'prefix' is supplied, use it instead of sys.base_prefix or >> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >> + """ >> +- if prefix is None: >> ++ if prefix is None and os.environ['STAGING_INCDIR'] != "": >> ++ prefix = os.environ['STAGING_INCDIR'].rstrip('include') >> ++ elif prefix is None: >> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >> + if os.name == "posix": >> + if python_build: >> +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >> + If 'prefix' is supplied, use it instead of sys.base_prefix or >> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >> + """ >> ++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] >> ++ if prefix is None and os.environ['STAGING_LIBDIR'] != "": >> ++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) >> ++ >> + if prefix is None: >> + if standard_lib: >> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >> +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >> + >> + if os.name == "posix": >> + libpython = os.path.join(prefix, >> +- "lib", "python" + get_python_version()) >> ++ lib_basename, "python" + get_python_version()) >> + if standard_lib: >> + return libpython >> + else: >> diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >> new file mode 100644 >> index 00000000000..24e67b4ca14 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >> @@ -0,0 +1,36 @@ >> +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001 >> +From: Andrei Gherzan <andrei@gherzan.ro> >> +Date: Mon, 28 Jan 2019 15:57:54 +0000 >> +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet >> + integrated in yocto so we skip the check for this module. Avoid a warning by >> + not adding this module to missing variable. >> + >> +Upstream-Status: Inappropriate [distribution] >> + >> +Also simply disable the tk module since its not in DEPENDS. >> +Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> >> + >> +--- >> + setup.py | 8 +++++--- >> + 1 file changed, 5 insertions(+), 3 deletions(-) >> + >> +diff --git a/setup.py b/setup.py >> +index fbec00d..b7a36a6 100644 >> +--- a/setup.py >> ++++ b/setup.py >> +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext): >> + self.extensions.extend(exts) >> + >> + # Call the method for detecting whether _tkinter can be compiled >> +- self.detect_tkinter(inc_dirs, lib_dirs) >> ++ # self.detect_tkinter(inc_dirs, lib_dirs) >> + >> +- if '_tkinter' not in [e.name for e in self.extensions]: >> +- missing.append('_tkinter') >> ++ # tkinter module will not be avalaible as yocto >> ++ # doesn't have tk integrated (yet) >> ++ #if '_tkinter' not in [e.name for e in self.extensions]: >> ++ # missing.append('_tkinter') >> + >> + # Build the _uuid module if possible >> + uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"]) >> diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >> new file mode 100644 >> index 00000000000..6c4ba54320b >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >> @@ -0,0 +1,32 @@ >> +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001 >> +From: Mark Hatle <mark.hatle@windriver.com> >> +Date: Wed, 21 Sep 2011 20:55:33 -0500 >> +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment >> + >> +Upstream-Status: Inappropriate [distribution] >> + >> +Signed-off-by: Mark Hatle <mark.hatle@windriver.com> >> + >> +--- >> + Lib/cgi.py | 11 +---------- >> + 1 file changed, 1 insertion(+), 10 deletions(-) >> + >> +diff --git a/Lib/cgi.py b/Lib/cgi.py >> +index 8cf6687..094c7b4 100755 >> +--- a/Lib/cgi.py >> ++++ b/Lib/cgi.py >> +@@ -1,13 +1,4 @@ >> +-#! /usr/local/bin/python >> +- >> +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is >> +-# intentionally NOT "/usr/bin/env python". On many systems >> +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI >> +-# scripts, and /usr/local/bin is the default directory where Python is >> +-# installed, so /usr/bin/env would be unable to find python. Granted, >> +-# binary installations by Linux vendors often install Python in >> +-# /usr/bin. So let those vendors patch cgi.py to match their choice >> +-# of installation. >> ++#! /usr/bin/env python >> + >> + """Support module for CGI (Common Gateway Interface) scripts. >> + >> diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >> new file mode 100755 >> index 00000000000..a1eace3f571 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >> @@ -0,0 +1,17 @@ >> +#!/usr/bin/env python3 >> +import sys >> +logfile = open(sys.argv[1]).read() >> + >> +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found") >> +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.") >> +if necessary_bits != -1: >> + print("%s" %(logfile[necessary_bits:to_find_bits])) >> + >> +failed_to_build = logfile.find("Failed to build these modules:") >> +if failed_to_build != -1: >> + failed_to_build_end = logfile.find("\n\n", failed_to_build) >> + print("%s" %(logfile[failed_to_build:failed_to_build_end])) >> + >> +if necessary_bits != -1 or failed_to_build != -1: >> + sys.exit(1) >> + >> diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >> new file mode 100644 >> index 00000000000..4da02a2991a >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >> @@ -0,0 +1,433 @@ >> +# This script is used as a bitbake task to create a new python manifest >> +# $ bitbake python -c create_manifest >> +# >> +# Our goal is to keep python-core as small as posible and add other python >> +# packages only when the user needs them, hence why we split upstream python >> +# into several packages. >> +# >> +# In a very simplistic way what this does is: >> +# Launch python and see specifically what is required for it to run at a minimum >> +# >> +# Go through the python-manifest file and launch a separate task for every single >> +# one of the files on each package, this task will check what was required for that >> +# specific module to run, these modules will be called dependencies. >> +# The output of such task will be a list of the modules or dependencies that were >> +# found for that file. >> +# >> +# Such output will be parsed by this script, we will look for each dependency on the >> +# manifest and if we find that another package already includes it, then we will add >> +# that package as an RDEPENDS to the package we are currently checking; in case we dont >> +# find the current dependency on any other package we will add it to the current package >> +# as part of FILES. >> +# >> +# >> +# This way we will create a new manifest from the data structure that was built during >> +# this process, on this new manifest each package will contain specifically only >> +# what it needs to run. >> +# >> +# There are some caveats which we try to deal with, such as repeated files on different >> +# packages, packages that include folders, wildcards, and special packages. >> +# Its also important to note that this method only works for python files, and shared >> +# libraries. Static libraries, header files and binaries need to be dealt with manually. >> +# >> +# This script differs from its python2 version mostly on how shared libraries are handled >> +# The manifest file for python3 has an extra field which contains the cached files for >> +# each package. >> +# Tha method to handle cached files does not work when a module includes a folder which >> +# itself contains the pycache folder, gladly this is almost never the case. >> +# >> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> >> + >> + >> +import sys >> +import subprocess >> +import json >> +import os >> +import collections >> + >> +# Get python version from ${PYTHON_MAJMIN} >> +pyversion = str(sys.argv[1]) >> + >> +# Hack to get native python search path (for folders), not fond of it but it works for now >> +pivot = 'recipe-sysroot-native' >> +for p in sys.path: >> + if pivot in p: >> + nativelibfolder = p[:p.find(pivot)+len(pivot)] >> + >> +# Empty dict to hold the whole manifest >> +new_manifest = collections.OrderedDict() >> + >> +# Check for repeated files, folders and wildcards >> +allfiles = [] >> +repeated = [] >> +wildcards = [] >> + >> +hasfolders = [] >> +allfolders = [] >> + >> +def isFolder(value): >> + value = value.replace('${PYTHON_MAJMIN}',pyversion) >> + if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): >> + return True >> + else: >> + return False >> + >> +def isCached(item): >> + if '__pycache__' in item: >> + return True >> + else: >> + return False >> + >> +def prepend_comments(comments, json_manifest): >> + with open(json_manifest, 'r+') as manifest: >> + json_contents = manifest.read() >> + manifest.seek(0, 0) >> + manifest.write(comments + json_contents) >> + >> +# Read existing JSON manifest >> +with open('python3-manifest.json') as manifest: >> + # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker >> + manifest_str = manifest.read() >> + json_start = manifest_str.find('# EOC') + 6 # EOC + \n >> + manifest.seek(0) >> + comments = manifest.read(json_start) >> + manifest_str = manifest.read() >> + old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >> + >> +# >> +# First pass to get core-package functionality, because we base everything on the fact that core is actually working >> +# Not exactly the same so it should not be a function >> +# >> + >> +print ('Getting dependencies for package: core') >> + >> + >> +# This special call gets the core dependencies and >> +# appends to the old manifest so it doesnt hurt what it >> +# currently holds. >> +# This way when other packages check for dependencies >> +# on the new core package, they will still find them >> +# even when checking the old_manifest >> + >> +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') >> +for coredep in output.split(): >> + coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') >> + if isCached(coredep): >> + if coredep not in old_manifest['core']['cached']: >> + old_manifest['core']['cached'].append(coredep) >> + else: >> + if coredep not in old_manifest['core']['files']: >> + old_manifest['core']['files'].append(coredep) >> + >> + >> +# The second step is to loop through the existing files contained in the core package >> +# according to the old manifest, identify if they are modules, or some other type >> +# of file that we cant import (directories, binaries, configs) in which case we >> +# can only assume they were added correctly (manually) so we ignore those and >> +# pass them to the manifest directly. >> + >> +for filedep in old_manifest['core']['files']: >> + if isFolder(filedep): >> + if isCached(filedep): >> + if filedep not in old_manifest['core']['cached']: >> + old_manifest['core']['cached'].append(filedep) >> + else: >> + if filedep not in old_manifest['core']['files']: >> + old_manifest['core']['files'].append(filedep) >> + continue >> + if '${bindir}' in filedep: >> + if filedep not in old_manifest['core']['files']: >> + old_manifest['core']['files'].append(filedep) >> + continue >> + if filedep == '': >> + continue >> + if '${includedir}' in filedep: >> + if filedep not in old_manifest['core']['files']: >> + old_manifest['core']['files'].append(filedep) >> + continue >> + >> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >> + pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] >> + >> + >> + # We now know that were dealing with a python module, so we can import it >> + # and check what its dependencies are. >> + # We launch a separate task for each module for deterministic behavior. >> + # Each module will only import what is necessary for it to work in specific. >> + # The output of each task will contain each module's dependencies >> + >> + print ('Getting dependencies for module: %s' % pymodule) >> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >> + print ('The following dependencies were found for module %s:\n' % pymodule) >> + print (output) >> + >> + >> + for pymodule_dep in output.split(): >> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >> + >> + if isCached(pymodule_dep): >> + if pymodule_dep not in old_manifest['core']['cached']: >> + old_manifest['core']['cached'].append(pymodule_dep) >> + else: >> + if pymodule_dep not in old_manifest['core']['files']: >> + old_manifest['core']['files'].append(pymodule_dep) >> + >> + >> +# At this point we are done with the core package. >> +# The old_manifest dictionary is updated only for the core package because >> +# all others will use this a base. >> + >> + >> +# To improve the script speed, we check which packages contain directories >> +# since we will be looping through (only) those later. >> +for pypkg in old_manifest: >> + for filedep in old_manifest[pypkg]['files']: >> + if isFolder(filedep): >> + print ('%s is a folder' % filedep) >> + if pypkg not in hasfolders: >> + hasfolders.append(pypkg) >> + if filedep not in allfolders: >> + allfolders.append(filedep) >> + >> + >> + >> +# This is the main loop that will handle each package. >> +# It works in a similar fashion than the step before, but >> +# we will now be updating a new dictionary that will eventually >> +# become the new manifest. >> +# >> +# The following loops though all packages in the manifest, >> +# through all files on each of them, and checks whether or not >> +# they are modules and can be imported. >> +# If they can be imported, then it checks for dependencies for >> +# each of them by launching a separate task. >> +# The output of that task is then parsed and the manifest is updated >> +# accordingly, wether it should add the module on FILES for the current package >> +# or if that module already belongs to another package then the current one >> +# will RDEPEND on it >> + >> +for pypkg in old_manifest: >> + # Use an empty dict as data structure to hold data for each package and fill it up >> + new_manifest[pypkg] = collections.OrderedDict() >> + new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] >> + new_manifest[pypkg]['rdepends'] = [] >> + new_manifest[pypkg]['files'] = [] >> + new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] >> + >> + # All packages should depend on core >> + if pypkg != 'core': >> + new_manifest[pypkg]['rdepends'].append('core') >> + new_manifest[pypkg]['cached'] = [] >> + >> + print('\n') >> + print('--------------------------') >> + print ('Handling package %s' % pypkg) >> + print('--------------------------') >> + >> + # Handle special cases, we assume that when they were manually added >> + # to the manifest we knew what we were doing. >> + special_packages = ['misc', 'modules', 'dev', 'tests'] >> + if pypkg in special_packages or 'staticdev' in pypkg: >> + print('Passing %s package directly' % pypkg) >> + new_manifest[pypkg] = old_manifest[pypkg] >> + continue >> + >> + for filedep in old_manifest[pypkg]['files']: >> + # We already handled core on the first pass, we can ignore it now >> + if pypkg == 'core': >> + if filedep not in new_manifest[pypkg]['files']: >> + new_manifest[pypkg]['files'].append(filedep) >> + continue >> + >> + # Handle/ignore what we cant import >> + if isFolder(filedep): >> + new_manifest[pypkg]['files'].append(filedep) >> + # Asyncio (and others) are both the package and the folder name, we should not skip those... >> + path,mod = os.path.split(filedep) >> + if mod != pypkg: >> + continue >> + if '${bindir}' in filedep: >> + if filedep not in new_manifest[pypkg]['files']: >> + new_manifest[pypkg]['files'].append(filedep) >> + continue >> + if filedep == '': >> + continue >> + if '${includedir}' in filedep: >> + if filedep not in new_manifest[pypkg]['files']: >> + new_manifest[pypkg]['files'].append(filedep) >> + continue >> + >> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >> + # We need to check if the imported module comes from another (e.g. sqlite3.dump) >> + path,pymodule = os.path.split(filedep) >> + path = os.path.basename(path) >> + pymodule = os.path.splitext(os.path.basename(pymodule))[0] >> + >> + # If this condition is met, it means we need to import it from another module >> + # or its the folder itself (e.g. unittest) >> + if path == pypkg: >> + if pymodule: >> + pymodule = path + '.' + pymodule >> + else: >> + pymodule = path >> + >> + >> + >> + # We now know that were dealing with a python module, so we can import it >> + # and check what its dependencies are. >> + # We launch a separate task for each module for deterministic behavior. >> + # Each module will only import what is necessary for it to work in specific. >> + # The output of each task will contain each module's dependencies >> + >> + print ('\nGetting dependencies for module: %s' % pymodule) >> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >> + print ('The following dependencies were found for module %s:\n' % pymodule) >> + print (output) >> + >> + reportFILES = [] >> + reportRDEPS = [] >> + >> + for pymodule_dep in output.split(): >> + >> + # Warning: This first part is ugly >> + # One of the dependencies that was found, could be inside of one of the folders included by another package >> + # We need to check if this happens so we can add the package containing the folder as an rdependency >> + # e.g. Folder encodings contained in codecs >> + # This would be solved if no packages included any folders >> + >> + # This can be done in two ways: >> + # 1 - We assume that if we take out the filename from the path we would get >> + # the folder string, then we would check if folder string is in the list of folders >> + # This would not work if a package contains a folder which contains another folder >> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >> + # folder_string would not match any value contained in the list of folders >> + # >> + # 2 - We do it the other way around, checking if the folder is contained in the path >> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >> + # is folder_string inside path/folder1/folder2/filename?, >> + # Yes, it works, but we waste a couple of milliseconds. >> + >> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >> + inFolders = False >> + for folder in allfolders: >> + # The module could have a directory named after it, e.g. xml, if we take out the filename from the path >> + # we'll end up with ${libdir}, and we want ${libdir}/xml >> + if isFolder(pymodule_dep): >> + check_path = pymodule_dep >> + else: >> + check_path = os.path.dirname(pymodule_dep) >> + if folder in check_path : >> + inFolders = True # Did we find a folder? >> + folderFound = False # Second flag to break inner for >> + # Loop only through packages which contain folders >> + for pypkg_with_folder in hasfolders: >> + if (folderFound == False): >> + # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) >> + for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: >> + if folder_dep == folder: >> + print ('%s folder found in %s' % (folder, pypkg_with_folder)) >> + folderFound = True >> + if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: >> + new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) >> + else: >> + break >> + >> + # A folder was found so we're done with this item, we can go on >> + if inFolders: >> + continue >> + >> + >> + >> + # No directories beyond this point >> + # We might already have this module on the dictionary since it could depend on a (previously checked) module >> + if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: >> + # Handle core as a special package, we already did it so we pass it to NEW data structure directly >> + if pypkg == 'core': >> + print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) >> + if pymodule_dep.endswith('*'): >> + wildcards.append(pymodule_dep) >> + if isCached(pymodule_dep): >> + new_manifest[pypkg]['cached'].append(pymodule_dep) >> + else: >> + new_manifest[pypkg]['files'].append(pymodule_dep) >> + >> + # Check for repeated files >> + if pymodule_dep not in allfiles: >> + allfiles.append(pymodule_dep) >> + else: >> + if pymodule_dep not in repeated: >> + repeated.append(pymodule_dep) >> + else: >> + >> + >> + # Last step: Figure out if we this belongs to FILES or RDEPENDS >> + # We check if this module is already contained on another package, so we add that one >> + # as an RDEPENDS, or if its not, it means it should be contained on the current >> + # package, and we should add it to FILES >> + for possible_rdep in old_manifest: >> + # Debug >> + # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) >> + if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: >> + # Since were nesting, we need to check its not the same pypkg >> + if(possible_rdep != pypkg): >> + if possible_rdep not in new_manifest[pypkg]['rdepends']: >> + # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs >> + reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) >> + new_manifest[pypkg]['rdepends'].append(possible_rdep) >> + break >> + else: >> + >> + # Since this module wasnt found on another package, it is not an RDEP, >> + # so we add it to FILES for this package. >> + # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) >> + if os.path.basename(pymodule_dep) != pypkg: >> + reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) >> + if isCached(pymodule_dep): >> + new_manifest[pypkg]['cached'].append(pymodule_dep) >> + else: >> + new_manifest[pypkg]['files'].append(pymodule_dep) >> + if pymodule_dep.endswith('*'): >> + wildcards.append(pymodule_dep) >> + if pymodule_dep not in allfiles: >> + allfiles.append(pymodule_dep) >> + else: >> + if pymodule_dep not in repeated: >> + repeated.append(pymodule_dep) >> + >> + print('\n') >> + print('#################################') >> + print('Summary for module %s' % pymodule) >> + print('FILES found for module %s:' % pymodule) >> + print(''.join(reportFILES)) >> + print('RDEPENDS found for module %s:' % pymodule) >> + print(''.join(reportRDEPS)) >> + print('#################################') >> + >> +print('The following FILES contain wildcards, please check if they are necessary') >> +print(wildcards) >> +print('The following FILES contain folders, please check if they are necessary') >> +print(hasfolders) >> + >> + >> +# Sort it just so it looks nicer >> +for pypkg in new_manifest: >> + new_manifest[pypkg]['files'].sort() >> + new_manifest[pypkg]['cached'].sort() >> + new_manifest[pypkg]['rdepends'].sort() >> + >> +# Create the manifest from the data structure that was built >> +with open('python3-manifest.json.new','w') as outfile: >> + json.dump(new_manifest,outfile, indent=4) >> + outfile.write('\n') >> + >> +prepend_comments(comments,'python3-manifest.json.new') >> + >> +if (repeated): >> + error_msg = '\n\nERROR:\n' >> + error_msg += 'The following files are repeated (contained in more than one package),\n' >> + error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' >> + error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' >> + error_msg += '\n'.join(repeated) >> + error_msg += '\n' >> + sys.exit(error_msg) >> + >> diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >> new file mode 100644 >> index 00000000000..fd12baad84e >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >> @@ -0,0 +1,146 @@ >> +# This script is launched on separate task for each python module >> +# It checks for dependencies for that specific module and prints >> +# them out, the output of this execution will have all dependencies >> +# for a specific module, which will be parsed an dealt on create_manifest.py >> +# >> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com> >> + >> +# We can get a log per module, for all the dependencies that were found, but its messy. >> +debug=False >> + >> +import sys >> + >> +# We can get a list of the modules which are currently required to run python >> +# so we run python-core and get its modules, we then import what we need >> +# and check what modules are currently running, if we substract them from the >> +# modules we had initially, we get the dependencies for the module we imported. >> + >> +# We use importlib to achieve this, so we also need to know what modules importlib needs >> +import importlib >> + >> +core_deps=set(sys.modules) >> + >> +def fix_path(dep_path): >> + import os >> + # We DONT want the path on our HOST system >> + pivot='recipe-sysroot-native' >> + dep_path=dep_path[dep_path.find(pivot)+len(pivot):] >> + >> + if '/usr/bin' in dep_path: >> + dep_path = dep_path.replace('/usr/bin''${bindir}') >> + >> + # Handle multilib, is there a better way? >> + if '/usr/lib32' in dep_path: >> + dep_path = dep_path.replace('/usr/lib32','${libdir}') >> + if '/usr/lib64' in dep_path: >> + dep_path = dep_path.replace('/usr/lib64','${libdir}') >> + if '/usr/lib' in dep_path: >> + dep_path = dep_path.replace('/usr/lib','${libdir}') >> + if '/usr/include' in dep_path: >> + dep_path = dep_path.replace('/usr/include','${includedir}') >> + if '__init__.' in dep_path: >> + dep_path = os.path.split(dep_path)[0] >> + return dep_path >> + >> + >> +# Module to import was passed as an argument >> +current_module = str(sys.argv[1]).rstrip() >> +if(debug==True): >> + log = open('log_%s' % current_module,'w') >> + log.write('Module %s generated the following dependencies:\n' % current_module) >> +try: >> + importlib.import_module('%s' % current_module) >> +except ImportError as e: >> + if (debug==True): >> + log.write('Module was not found') >> + pass >> + >> + >> +# Get current module dependencies, dif will contain a list of specific deps for this module >> +module_deps=set(sys.modules) >> + >> +# We handle the core package (1st pass on create_manifest.py) as a special case >> +if current_module == 'python-core-package': >> + dif = core_deps >> +else: >> + # We know this is not the core package, so there must be a difference. >> + dif = module_deps-core_deps >> + >> + >> +# Check where each dependency came from >> +for item in dif: >> + dep_path='' >> + try: >> + if (debug==True): >> + log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') >> + dep_path = sys.modules['%s' % item].__file__ >> + except AttributeError as e: >> + # Deals with thread (builtin module) not having __file__ attribute >> + if debug==True: >> + log.write(item + ' ') >> + log.write(str(e)) >> + log.write('\n') >> + pass >> + except NameError as e: >> + # Deals with NameError: name 'dep_path' is not defined >> + # because module is not found (wasn't compiled?), e.g. bddsm >> + if (debug==True): >> + log.write(item+' ') >> + log.write(str(e)) >> + pass >> + >> + # Site-customize is a special case since we (OpenEmbedded) put it there manually >> + if 'sitecustomize' in dep_path: >> + dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' >> + # Prints out result, which is what will be used by create_manifest >> + print (dep_path) >> + continue >> + >> + dep_path = fix_path(dep_path) >> + >> + import sysconfig >> + soabi=sysconfig.get_config_var('SOABI') >> + # Check if its a shared library and deconstruct it >> + if soabi in dep_path: >> + if (debug==True): >> + log.write('Shared library found in %s' % dep_path) >> + dep_path = dep_path.replace(soabi,'*') >> + print (dep_path) >> + continue >> + >> + if (debug==True): >> + log.write(dep_path+'\n') >> + # Prints out result, which is what will be used by create_manifest >> + print (dep_path) >> + >> + >> + import imp >> + cpython_tag = imp.get_tag() >> + cached='' >> + # Theres no naive way to find *.pyc files on python3 >> + try: >> + if (debug==True): >> + log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') >> + cached = sys.modules['%s' % item].__cached__ >> + except AttributeError as e: >> + # Deals with thread (builtin module) not having __cached__ attribute >> + if debug==True: >> + log.write(item + ' ') >> + log.write(str(e)) >> + log.write('\n') >> + pass >> + except NameError as e: >> + # Deals with NameError: name 'cached' is not defined >> + if (debug==True): >> + log.write(item+' ') >> + log.write(str(e)) >> + pass >> + if cached is not None: >> + if (debug==True): >> + log.write(cached) >> + cached = fix_path(cached) >> + cached = cached.replace(cpython_tag,'*') >> + print (cached) >> + >> +if debug==True: >> + log.close() >> diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch >> new file mode 100644 >> index 00000000000..f23b8b7df06 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch >> @@ -0,0 +1,46 @@ >> +python-config: Revert to using distutils.sysconfig >> + >> +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in >> + >> +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig >> + >> +affect the native runtime as well as cross building. Use the old, patched >> +implementation which returns paths in the staging directory and for the target, >> +as appropriate. >> + >> +Upstream-Status: Inappropriate [Embedded Specific] >> + >> +Signed-off-by: Tyler Hall <tylerwhall@gmail.com> >> +: >> +Index: Python-3.3.3/Misc/python-config.in >> +=================================================================== >> +--- Python-3.3.3.orig/Misc/python-config.in >> ++++ Python-3.3.3/Misc/python-config.in >> +@@ -4,7 +4,7 @@ >> + import getopt >> + import os >> + import sys >> +-import sysconfig >> ++from distutils import sysconfig >> + >> + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', >> + 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] >> +@@ -32,14 +32,14 @@ if '--help' in opt_flags: >> + >> + for opt in opt_flags: >> + if opt == '--prefix': >> +- print(sysconfig.get_config_var('prefix')) >> ++ print(sysconfig.PREFIX) >> + >> + elif opt == '--exec-prefix': >> +- print(sysconfig.get_config_var('exec_prefix')) >> ++ print(sysconfig.EXEC_PREFIX) >> + >> + elif opt in ('--includes', '--cflags'): >> +- flags = ['-I' + sysconfig.get_path('include'), >> +- '-I' + sysconfig.get_path('platinclude')] >> ++ flags = ['-I' + sysconfig.get_python_inc(), >> ++ '-I' + sysconfig.get_python_inc(plat_specific=True)] >> + if opt == '--cflags': >> + flags.extend(getvar('CFLAGS').split()) >> + print(' '.join(flags)) >> diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >> new file mode 100644 >> index 00000000000..82c7075f0e7 >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >> @@ -0,0 +1,1227 @@ >> +# DO NOT (entirely) modify this file manually, please read. >> +# >> +# IMPORTANT NOTE: >> +# Please keep in mind that the create_manifest task relies on the fact the the >> +# target and native Python packages are the same, and it also needs to be executed >> +# with a fully working native package (with all the PACKAGECONFIGs enabled and all >> +# and all the modules should be working, check log.do_compile), otherwise the script >> +# will fail to find dependencies correctly, this note is valid either if you are >> +# upgrading to a new Python version or adding a new package. >> +# >> +# >> +# If you are adding a new package please follow the next steps: >> +# How to add a new package: >> +# - If a user wants to add a new package all that has to be done is: >> +# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, >> +# fill up the SUMMARY section as well, the script should handle all the rest. >> +# >> +# Real example: >> +# We want to add a web browser package, including the file webbrowser.py >> +# which at the moment is on python3-misc. >> +# "webbrowser": { >> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], >> +# "rdepends": [], >> +# "summary": "Python Web Browser support"} >> +# >> +# * Note that the rdepends field was left empty >> +# >> +# We run $ bitbake python3 -c create_manifest and the resulting manifest >> +# should be completed after a few seconds, showing something like: >> +# "webbrowser": { >> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], >> +# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], >> +# "summary": "Python Web Browser support"} >> +# >> +# >> +# If you are upgrading Python to a new version please follow the next steps: >> +# After each Python upgrade, the create_manifest task should be executed, because we >> +# don't control what changes on upstream Python, so, some module dependency >> +# might have changed without us realizing it, a certain module can either have >> +# more or less dependencies, or could be depending on a new file that was just >> +# created on the new release and for obvious reasons we wouldn't have it on our >> +# old manifest, all of these issues would cause runtime errors on our system. >> +# >> +# - Upgrade both the native and target Python packages to a new version >> +# - Run the create_manifest task for the target Python package as its shown below: >> +# >> +# $ bitbake python3 -c create_manifest >> +# >> +# This will automatically replace your manifest file located under the Python directory >> +# with an new one, which contains the new dependencies (if any). >> +# >> +# Several things could have gone wrong here, I will try to explain a few: >> +# >> +# a) A new file was introduced on this release, e.g. sha3*.so: >> +# The task will check what its needed to import every module, more than one module would >> +# would probably depend on sha3*.so, although only one module should contain it. >> +# >> +# After running the task, the new manifest will have the sha3*.so file on more than one >> +# module, you need to manually decide which one of them should get it and delete it from >> +# the others, for example sha3*.so should likely be on ${PN}-crypt. >> +# Once you have deleted from the others you need to run the create_manifest task again, >> +# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be >> +# good to go. >> +# >> +# b) The native package wasn't built correctly and its missing a certain module: >> +# As mentioned before, you need to make sure the native package was built with all the modules >> +# because it is used as base to build the manifest file, you need to manually check log.do_compile >> +# since it won't error out the compile function if its only missing a couple of modules. >> +# >> +# e.g. missing the _uuid module, log.do_compile would show the following: >> +# Python build finished successfully! >> +# The necessary bits to build these optional modules were not found: >> +# _uuid >> +# >> +# What will happen here is that the new manifest would not be aware that the _uuid module exists, so >> +# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on >> +# the misc package (which is where any file that doesn't belong anywhere else ends up). >> +# >> +# This will eventually cause runtime errors on our system if we don't include the misc package on >> +# on our image, because the _uuid files will be missing. >> +# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be >> +# detected correctly along with its dependencies, and we will get a working manifest. >> +# >> +# This is the reason why it is important to make sure we have a fully working native build, >> +# so we can avoid these errors. >> +# >> +# >> +# >> +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST >> +# EOC >> +{ >> + "tests": { >> + "summary": "Python test suite", >> + "rdepends": [ >> + "core", >> + "modules" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/*/test", >> + "${libdir}/python${PYTHON_MAJMIN}/*/tests", >> + "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", >> + "${libdir}/python${PYTHON_MAJMIN}/test" >> + ], >> + "cached": [] >> + }, >> + "2to3": { >> + "summary": "Python automated Python 2 to 3 code translator", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${bindir}/2to3-*", >> + "${libdir}/python${PYTHON_MAJMIN}/lib2to3" >> + ], >> + "cached": [] >> + }, >> + "asyncio": { >> + "summary": "Python Asynchronous I/", >> + "rdepends": [ >> + "core", >> + "io", >> + "logging", >> + "netclient", >> + "numbers", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/asyncio", >> + "${libdir}/python${PYTHON_MAJMIN}/concurrent", >> + "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so" >> + ], >> + "cached": [] >> + }, >> + "audio": { >> + "summary": "Python Audio Handling", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/chunk.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sunau.py", >> + "${libdir}/python${PYTHON_MAJMIN}/wave.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" >> + ] >> + }, >> + "codecs": { >> + "summary": "Python codec", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" >> + ] >> + }, >> + "compile": { >> + "summary": "Python bytecode compilation support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/compileall.py", >> + "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" >> + ] >> + }, >> + "compression": { >> + "summary": "Python high-level compression support", >> + "rdepends": [ >> + "core", >> + "shell", >> + "unixadmin" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_compression.py", >> + "${libdir}/python${PYTHON_MAJMIN}/bz2.py", >> + "${libdir}/python${PYTHON_MAJMIN}/gzip.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lzma.py", >> + "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", >> + "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" >> + ] >> + }, >> + "core": { >> + "summary": "Python interpreter and core modules", >> + "rdepends": [], >> + "files": [ >> + "${bindir}/python*[!-config]", >> + "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", >> + "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", >> + "${libdir}/python${PYTHON_MAJMIN}/UserList.py", >> + "${libdir}/python${PYTHON_MAJMIN}/UserString.py", >> + "${libdir}/python${PYTHON_MAJMIN}/__future__.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", >> + "${libdir}/python${PYTHON_MAJMIN}/abc.py", >> + "${libdir}/python${PYTHON_MAJMIN}/argparse.py", >> + "${libdir}/python${PYTHON_MAJMIN}/ast.py", >> + "${libdir}/python${PYTHON_MAJMIN}/bisect.py", >> + "${libdir}/python${PYTHON_MAJMIN}/code.py", >> + "${libdir}/python${PYTHON_MAJMIN}/codecs.py", >> + "${libdir}/python${PYTHON_MAJMIN}/codeop.py", >> + "${libdir}/python${PYTHON_MAJMIN}/collections", >> + "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", >> + "${libdir}/python${PYTHON_MAJMIN}/configparser.py", >> + "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/copy.py", >> + "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", >> + "${libdir}/python${PYTHON_MAJMIN}/csv.py", >> + "${libdir}/python${PYTHON_MAJMIN}/dis.py", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", >> + "${libdir}/python${PYTHON_MAJMIN}/enum.py", >> + "${libdir}/python${PYTHON_MAJMIN}/functools.py", >> + "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", >> + "${libdir}/python${PYTHON_MAJMIN}/getopt.py", >> + "${libdir}/python${PYTHON_MAJMIN}/gettext.py", >> + "${libdir}/python${PYTHON_MAJMIN}/heapq.py", >> + "${libdir}/python${PYTHON_MAJMIN}/imp.py", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", >> + "${libdir}/python${PYTHON_MAJMIN}/inspect.py", >> + "${libdir}/python${PYTHON_MAJMIN}/io.py", >> + "${libdir}/python${PYTHON_MAJMIN}/keyword.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/linecache.py", >> + "${libdir}/python${PYTHON_MAJMIN}/locale.py", >> + "${libdir}/python${PYTHON_MAJMIN}/new.py", >> + "${libdir}/python${PYTHON_MAJMIN}/opcode.py", >> + "${libdir}/python${PYTHON_MAJMIN}/operator.py", >> + "${libdir}/python${PYTHON_MAJMIN}/optparse.py", >> + "${libdir}/python${PYTHON_MAJMIN}/os.py", >> + "${libdir}/python${PYTHON_MAJMIN}/platform.py", >> + "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", >> + "${libdir}/python${PYTHON_MAJMIN}/re.py", >> + "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", >> + "${libdir}/python${PYTHON_MAJMIN}/selectors.py", >> + "${libdir}/python${PYTHON_MAJMIN}/signal.py", >> + "${libdir}/python${PYTHON_MAJMIN}/site.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", >> + "${libdir}/python${PYTHON_MAJMIN}/stat.py", >> + "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", >> + "${libdir}/python${PYTHON_MAJMIN}/struct.py", >> + "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", >> + "${libdir}/python${PYTHON_MAJMIN}/symbol.py", >> + "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", >> + "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", >> + "${libdir}/python${PYTHON_MAJMIN}/threading.py", >> + "${libdir}/python${PYTHON_MAJMIN}/token.py", >> + "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", >> + "${libdir}/python${PYTHON_MAJMIN}/traceback.py", >> + "${libdir}/python${PYTHON_MAJMIN}/types.py", >> + "${libdir}/python${PYTHON_MAJMIN}/warnings.py", >> + "${libdir}/python${PYTHON_MAJMIN}/weakref.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", >> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" >> + ] >> + }, >> + "crypt": { >> + "summary": "Python basic cryptographic and hashing support", >> + "rdepends": [ >> + "core", >> + "math", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/crypt.py", >> + "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" >> + ] >> + }, >> + "ctypes": { >> + "summary": "Python C types support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/ctypes", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" >> + ], >> + "cached": [] >> + }, >> + "curses": { >> + "summary": "Python curses support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/curses", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" >> + ], >> + "cached": [] >> + }, >> + "datetime": { >> + "summary": "Python calendar and time support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", >> + "${libdir}/python${PYTHON_MAJMIN}/calendar.py", >> + "${libdir}/python${PYTHON_MAJMIN}/datetime.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" >> + ] >> + }, >> + "db": { >> + "summary": "Python file-based database support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/dbm", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" >> + ], >> + "cached": [] >> + }, >> + "debugger": { >> + "summary": "Python debugger", >> + "rdepends": [ >> + "core", >> + "pprint", >> + "shell", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/bdb.py", >> + "${libdir}/python${PYTHON_MAJMIN}/pdb.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" >> + ] >> + }, >> + "dev": { >> + "cached": [], >> + "files": [ >> + "${base_libdir}/*.a", >> + "${base_libdir}/*.o", >> + "${bindir}/python*-config", >> + "${datadir}/aclocal", >> + "${datadir}/pkgconfig", >> + "${includedir}", >> + "${libdir}/*.a", >> + "${libdir}/*.la", >> + "${libdir}/*.o", >> + "${libdir}/lib*${SOLIBSDEV}", >> + "${libdir}/pkgconfig", >> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", >> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" >> + ], >> + "rdepends": [ >> + "core" >> + ], >> + "summary": "Python development package" >> + }, >> + "difflib": { >> + "summary": "Python helpers for computing deltas between objects", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/difflib.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" >> + ] >> + }, >> + "distutils-staticdev": { >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" >> + ], >> + "rdepends": [ >> + "distutils" >> + ], >> + "summary": "Python distribution utilities (static libraries)" >> + }, >> + "distutils": { >> + "summary": "Python Distribution Utilities", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/distutils" >> + ], >> + "cached": [] >> + }, >> + "doctest": { >> + "summary": "Python framework for running examples in docstrings", >> + "rdepends": [ >> + "core", >> + "debugger", >> + "difflib", >> + "logging", >> + "pprint", >> + "shell", >> + "stringold", >> + "unittest" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/doctest.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" >> + ] >> + }, >> + "email": { >> + "summary": "Python email support", >> + "rdepends": [ >> + "core", >> + "crypt", >> + "datetime", >> + "io", >> + "math", >> + "netclient" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/email", >> + "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" >> + ] >> + }, >> + "fcntl": { >> + "summary": "Python's fcntl interface", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" >> + ], >> + "cached": [] >> + }, >> + "gdbm": { >> + "summary": "Python GNU database support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" >> + ], >> + "cached": [] >> + }, >> + "html": { >> + "summary": "Python HTML processing support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/formatter.py", >> + "${libdir}/python${PYTHON_MAJMIN}/html" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" >> + ] >> + }, >> + "idle": { >> + "summary": "Python Integrated Development Environment", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${bindir}/idle*", >> + "${libdir}/python${PYTHON_MAJMIN}/idlelib" >> + ], >> + "cached": [] >> + }, >> + "image": { >> + "summary": "Python graphical image handling", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", >> + "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" >> + ] >> + }, >> + "io": { >> + "summary": "Python low-level I/O", >> + "rdepends": [ >> + "compression", >> + "core", >> + "crypt", >> + "math", >> + "netclient", >> + "shell", >> + "unixadmin" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", >> + "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/pipes.py", >> + "${libdir}/python${PYTHON_MAJMIN}/socket.py", >> + "${libdir}/python${PYTHON_MAJMIN}/ssl.py", >> + "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" >> + ] >> + }, >> + "json": { >> + "summary": "Python JSON support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/json", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" >> + ], >> + "cached": [] >> + }, >> + "logging": { >> + "summary": "Python logging support", >> + "rdepends": [ >> + "core", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/logging" >> + ], >> + "cached": [] >> + }, >> + "mailbox": { >> + "summary": "Python mailbox format support", >> + "rdepends": [ >> + "core", >> + "crypt", >> + "datetime", >> + "email", >> + "fcntl", >> + "io", >> + "math", >> + "mime", >> + "netclient", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" >> + ] >> + }, >> + "math": { >> + "summary": "Python math support", >> + "rdepends": [ >> + "core", >> + "crypt" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/random.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" >> + ] >> + }, >> + "mime": { >> + "summary": "Python MIME handling APIs", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/quopri.py", >> + "${libdir}/python${PYTHON_MAJMIN}/uu.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" >> + ] >> + }, >> + "mmap": { >> + "summary": "Python memory-mapped file support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" >> + ], >> + "cached": [] >> + }, >> + "modules": { >> + "cached": [], >> + "files": [], >> + "rdepends": [ >> + "2to3", >> + "asyncio", >> + "audio", >> + "codecs", >> + "compile", >> + "compression", >> + "core", >> + "crypt", >> + "ctypes", >> + "curses", >> + "datetime", >> + "db", >> + "debugger", >> + "difflib", >> + "distutils", >> + "doctest", >> + "email", >> + "fcntl", >> + "html", >> + "idle", >> + "image", >> + "io", >> + "json", >> + "logging", >> + "mailbox", >> + "math", >> + "mime", >> + "mmap", >> + "multiprocessing", >> + "netclient", >> + "netserver", >> + "numbers", >> + "pickle", >> + "pkgutil", >> + "plistlib", >> + "pprint", >> + "profile", >> + "pydoc", >> + "resource", >> + "runpy", >> + "shell", >> + "smtpd", >> + "sqlite3", >> + "stringold", >> + "syslog", >> + "terminal", >> + "threading", >> + "tkinter", >> + "typing", >> + "unittest", >> + "unixadmin", >> + "venv", >> + "xml", >> + "xmlrpc" >> + ], >> + "summary": "All Python modules" >> + }, >> + "multiprocessing": { >> + "summary": "Python multiprocessing support", >> + "rdepends": [ >> + "core", >> + "io", >> + "pickle" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" >> + ], >> + "cached": [] >> + }, >> + "netclient": { >> + "summary": "Python Internet Protocol clients", >> + "rdepends": [ >> + "core", >> + "crypt", >> + "datetime", >> + "email", >> + "io", >> + "math", >> + "mime", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/base64.py", >> + "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/hmac.py", >> + "${libdir}/python${PYTHON_MAJMIN}/http", >> + "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", >> + "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/poplib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", >> + "${libdir}/python${PYTHON_MAJMIN}/urllib", >> + "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", >> + "${libdir}/python${PYTHON_MAJMIN}/uuid.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" >> + ] >> + }, >> + "netserver": { >> + "summary": "Python Internet Protocol servers", >> + "rdepends": [ >> + "compression", >> + "core", >> + "crypt", >> + "datetime", >> + "email", >> + "html", >> + "io", >> + "math", >> + "mime", >> + "netclient", >> + "shell", >> + "stringold", >> + "unixadmin" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/cgi.py", >> + "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" >> + ] >> + }, >> + "numbers": { >> + "summary": "Python number APIs", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", >> + "${libdir}/python${PYTHON_MAJMIN}/contextvars.py", >> + "${libdir}/python${PYTHON_MAJMIN}/decimal.py", >> + "${libdir}/python${PYTHON_MAJMIN}/fractions.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/numbers.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" >> + ] >> + }, >> + "pickle": { >> + "summary": "Python serialisation/persistence support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/pickle.py", >> + "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", >> + "${libdir}/python${PYTHON_MAJMIN}/shelve.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" >> + ] >> + }, >> + "pkgutil": { >> + "summary": "Python package extension utility support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" >> + ] >> + }, >> + "plistlib": { >> + "summary": "Generate and parse Mac OS X .plist files", >> + "rdepends": [ >> + "core", >> + "datetime", >> + "xml" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" >> + ] >> + }, >> + "pprint": { >> + "summary": "Python pretty-print support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/pprint.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" >> + ] >> + }, >> + "profile": { >> + "summary": "Python basic performance profiling support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/profile.py", >> + "${libdir}/python${PYTHON_MAJMIN}/pstats.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" >> + ] >> + }, >> + "pydoc": { >> + "summary": "Python interactive help support", >> + "rdepends": [ >> + "core", >> + "netclient", >> + "pkgutil" >> + ], >> + "files": [ >> + "${bindir}/pydoc*", >> + "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", >> + "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" >> + ] >> + }, >> + "resource": { >> + "summary": "Python resource control interface", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" >> + ], >> + "cached": [] >> + }, >> + "runpy": { >> + "summary": "Python helper for locating/executing scripts in module namespace", >> + "rdepends": [ >> + "core", >> + "pkgutil" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/runpy.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" >> + ] >> + }, >> + "shell": { >> + "summary": "Python shell-like functionality", >> + "rdepends": [ >> + "compression", >> + "core", >> + "stringold", >> + "unixadmin" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/cmd.py", >> + "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", >> + "${libdir}/python${PYTHON_MAJMIN}/glob.py", >> + "${libdir}/python${PYTHON_MAJMIN}/shlex.py", >> + "${libdir}/python${PYTHON_MAJMIN}/shutil.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" >> + ] >> + }, >> + "smtpd": { >> + "summary": "Python Simple Mail Transport Daemon", >> + "rdepends": [ >> + "core", >> + "crypt", >> + "datetime", >> + "email", >> + "io", >> + "math", >> + "mime", >> + "netclient", >> + "stringold" >> + ], >> + "files": [ >> + "${bindir}/smtpd.py", >> + "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", >> + "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", >> + "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" >> + ] >> + }, >> + "sqlite3": { >> + "summary": "Python Sqlite3 database support", >> + "rdepends": [ >> + "core", >> + "datetime" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/sqlite3" >> + ], >> + "cached": [] >> + }, >> + "stringold": { >> + "summary": "Python string APIs [deprecated]", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/string.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" >> + ] >> + }, >> + "syslog": { >> + "summary": "Python syslog interface", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" >> + ], >> + "cached": [] >> + }, >> + "terminal": { >> + "summary": "Python terminal controlling support", >> + "rdepends": [ >> + "core", >> + "io" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/pty.py", >> + "${libdir}/python${PYTHON_MAJMIN}/tty.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" >> + ] >> + }, >> + "threading": { >> + "summary": "Python threading & synchronization support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", >> + "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/queue.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" >> + ] >> + }, >> + "tkinter": { >> + "summary": "Python Tcl/Tk bindings", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/tkinter" >> + ], >> + "cached": [] >> + }, >> + "typing": { >> + "summary": "Python typing support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/typing.py" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" >> + ] >> + }, >> + "unittest": { >> + "summary": "Python unit testing framework", >> + "rdepends": [ >> + "core", >> + "difflib", >> + "logging", >> + "pprint", >> + "shell", >> + "stringold" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/unittest", >> + "${libdir}/python${PYTHON_MAJMIN}/unittest/", >> + "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" >> + ], >> + "cached": [] >> + }, >> + "unixadmin": { >> + "summary": "Python Unix administration support", >> + "rdepends": [ >> + "core", >> + "io" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/getpass.py", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" >> + ], >> + "cached": [ >> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" >> + ] >> + }, >> + "venv": { >> + "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", >> + "rdepends": [ >> + "compression", >> + "core", >> + "logging", >> + "shell", >> + "stringold", >> + "unixadmin" >> + ], >> + "files": [ >> + "${bindir}/pyvenv*", >> + "${libdir}/python${PYTHON_MAJMIN}/venv" >> + ], >> + "cached": [] >> + }, >> + "xml": { >> + "summary": "Python basic XML support", >> + "rdepends": [ >> + "core" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", >> + "${libdir}/python${PYTHON_MAJMIN}/xml" >> + ], >> + "cached": [] >> + }, >> + "xmlrpc": { >> + "summary": "Python XML-RPC support", >> + "rdepends": [ >> + "core", >> + "xml" >> + ], >> + "files": [ >> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", >> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" >> + ], >> + "cached": [] >> + } >> +} >> diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest >> new file mode 100644 >> index 00000000000..3863c6d314f >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest >> @@ -0,0 +1,3 @@ >> +#!/bin/sh >> + >> +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' >> diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >> new file mode 100644 >> index 00000000000..61fa12ffe8d >> --- /dev/null >> +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >> @@ -0,0 +1,281 @@ >> +SUMMARY = "The Python Programming Language" >> +HOMEPAGE = "http://www.python.org" >> +LICENSE = "PSFv2" >> +SECTION = "devel/python" >> + >> +LIC_FILES_CHKSUM = "file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754" >> + >> +SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ >> + file://run-ptest \ >> + file://create_manifest3.py \ >> + file://get_module_deps3.py \ >> + file://python3-manifest.json \ >> + file://check_build_completeness.py \ >> + file://cgi_py.patch \ >> + file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \ >> + ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \ >> + file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \ >> + file://python-config.patch \ >> + file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \ >> + " >> + >> +SRC_URI_append_class-native = " \ >> + file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ >> + file://12-distutils-prefix-is-inside-staging-area.patch \ >> + " >> + >> +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb" >> +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb" >> + >> +# exclude pre-releases for both python 2.x and 3.x >> +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" >> + >> +CVE_PRODUCT = "python" >> + >> +PYTHON_MAJMIN = "3.7" >> +PYTHON_BINABI = "${PYTHON_MAJMIN}m" >> + >> +S = "${WORKDIR}/Python-${PV}" >> + >> +BBCLASSEXTEND = "native nativesdk" >> + >> +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives >> + >> +MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}" >> + >> +ALTERNATIVE_${PN}-dev = "python-config" >> +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" >> +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" >> + >> + >> +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2" >> +DEPENDS_append_class-target = " python3-native" >> +DEPENDS_append_class-nativesdk = " python3-native" >> + >> +EXTRA_OECONF = " --without-ensurepip --enable-shared" >> +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}" >> + >> + >> +EXTRANATIVEPATH += "python3-native" >> + >> +CACHED_CONFIGUREVARS = " \ >> + ac_cv_file__dev_ptmx=yes \ >> + ac_cv_file__dev_ptc=no \ >> +" >> + >> +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" >> +PACKAGECONFIG_class-native ??= "readline" >> +PACKAGECONFIG_class-nativesdk ??= "readline" >> +PACKAGECONFIG[readline] = ",,readline" >> +# Use profile guided optimisation by running PyBench inside qemu-user >> +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native" >> +PACKAGECONFIG[tk] = ",,tk" >> + >> +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid" >> + >> +EXTRA_OEMAKE = '\ >> + STAGING_LIBDIR=${STAGING_LIBDIR} \ >> + STAGING_INCDIR=${STAGING_INCDIR} \ >> +' >> + >> +do_compile_prepend_class-target() { >> + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then >> + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" >> + cat >pgo-wrapper <<EOF >> +#!/bin/sh >> +cd ${B} >> +$qemu_binary "\$@" >> +EOF >> + chmod +x pgo-wrapper >> + fi >> +} >> + >> +do_install_prepend() { >> + ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile >> +} >> + >> +do_install_append_class-target() { >> + oe_multilib_header python${PYTHON_BINABI}/pyconfig.h >> +} >> + >> +do_install_append_class-native() { >> + # Make sure we use /usr/bin/env python >> + for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do >> + sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT >> + done >> + # Add a symlink to the native Python so that scripts can just invoke >> + # "nativepython" and get the right one without needing absolute paths >> + # (these often end up too long for the #! parser in the kernel as the >> + # buffer is 128 bytes long). >> + ln -s python3-native/python3 ${D}${bindir}/nativepython3 >> +} >> + >> +do_install_append() { >> + mkdir -p ${D}${libdir}/python-sysconfigdata >> + sysconfigfile=`find ${D} -name _sysconfig*.py` >> + cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >> + >> + sed -i \ >> + -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \ >> + -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >> + -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >> + ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >> +} >> + >> +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py" >> +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" >> + >> +py_package_preprocess () { >> + # Remove references to buildmachine paths in target Makefile and _sysconfigdata >> + sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ >> + -e 's|${DEBUG_PREFIX_MAP}||g' \ >> + -e 's:${HOSTTOOLS_DIR}/::g' \ >> + -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ >> + -e 's:${RECIPE_SYSROOT}::g' \ >> + -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ >> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \ >> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \ >> + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config >> + >> + # Recompile _sysconfigdata after modifying it >> + cd ${PKGD} >> + sysconfigfile=`find . -name _sysconfigdata_*.py` >> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >> + -c "from py_compile import compile; compile('$sysconfigfile')" >> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)" >> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)" >> + cd - >> + >> + mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} >> + >> + #Remove the unneeded copy of target sysconfig data >> + rm -rf ${PKGD}/${libdir}/python-sysconfigdata >> +} >> + >> +# We want bytecode precompiled .py files (.pyc's) by default >> +# but the user may set it on their own conf >> +INCLUDE_PYCS ?= "1" >> + >> +python(){ >> + import collections, json >> + >> + filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') >> + # This python changes the datastore based on the contents of a file, so mark >> + # that dependency. >> + bb.parse.mark_dependency(d, filename) >> + >> + with open(filename) as manifest_file: >> + manifest_str = manifest_file.read() >> + json_start = manifest_str.find('# EOC') + 6 >> + manifest_file.seek(json_start) >> + manifest_str = manifest_file.read() >> + python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >> + >> + # First set RPROVIDES for -native case >> + # Hardcoded since it cant be python3-native-foo, should be python3-foo-native >> + pn = 'python3' >> + rprovides = d.getVar('RPROVIDES').split() >> + >> + for key in python_manifest: >> + pypackage = pn + '-' + key + '-native' >> + if pypackage not in rprovides: >> + rprovides.append(pypackage) >> + >> + d.setVar('RPROVIDES_class-native', ' '.join(rprovides)) >> + >> + # Then work on the target >> + include_pycs = d.getVar('INCLUDE_PYCS') >> + >> + packages = d.getVar('PACKAGES').split() >> + pn = d.getVar('PN') >> + >> + newpackages=[] >> + for key in python_manifest: >> + pypackage= pn + '-' + key >> + >> + if pypackage not in packages: >> + # We need to prepend, otherwise python-misc gets everything >> + # so we use a new variable >> + newpackages.append(pypackage) >> + >> + # "Build" python's manifest FILES, RDEPENDS and SUMMARY >> + d.setVar('FILES_' + pypackage, '') >> + for value in python_manifest[key]['files']: >> + d.appendVar('FILES_' + pypackage, ' ' + value) >> + >> + # Add cached files >> + if include_pycs == '1': >> + for value in python_manifest[key]['cached']: >> + d.appendVar('FILES_' + pypackage, ' ' + value) >> + >> + for value in python_manifest[key]['rdepends']: >> + # Make it work with or without $PN >> + if '${PN}' in value: >> + value=value.split('-')[1] >> + d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) >> + d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) >> + >> + # Prepending so to avoid python-misc getting everything >> + packages = newpackages + packages >> + d.setVar('PACKAGES', ' '.join(packages)) >> + d.setVar('ALLOW_EMPTY_${PN}-modules', '1') >> +} >> + >> +# Files needed to create a new manifest >> + >> +do_create_manifest() { >> + # This task should be run with every new release of Python. >> + # We must ensure that PACKAGECONFIG enables everything when creating >> + # a new manifest, this is to base our new manifest on a complete >> + # native python build, containing all dependencies, otherwise the task >> + # wont be able to find the required files. >> + # e.g. BerkeleyDB is an optional build dependency so it may or may not >> + # be present, we must ensure it is. >> + >> + cd ${WORKDIR} >> + # This needs to be executed by python-native and NOT by HOST's python >> + nativepython3 create_manifest3.py ${PYTHON_MAJMIN} >> + cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json >> +} >> + >> +# bitbake python -c create_manifest >> +addtask do_create_manifest >> + >> +# Make sure we have native python ready when we create a new manifest >> +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" >> +do_create_manifest[depends] += "python3:do_patch" >> + >> +# manual dependency additions >> +RPROVIDES_${PN}-modules = "${PN}" >> +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" >> +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" >> + >> +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" >> +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" >> +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" >> + >> +# provide python-pyvenv from python3-venv >> +RPROVIDES_${PN}-venv += "python3-pyvenv" >> + >> +# package libpython3 >> +PACKAGES =+ "libpython3 libpython3-staticdev" >> +FILES_libpython3 = "${libdir}/libpython*.so.*" >> +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a" >> +INSANE_SKIP_${PN}-dev += "dev-elf" >> + >> +# catch all the rest (unsorted) >> +PACKAGES += "${PN}-misc" >> +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs" >> +RDEPENDS_${PN}-modules += "python3-misc" >> +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" >> + >> +# catch manpage >> +PACKAGES += "${PN}-man" >> +FILES_${PN}-man = "${datadir}/man" >> + >> +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" >> +RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" >> +RDEPENDS_${PN}-dev = "" >> + >> -- >> 2.17.1 >> >> -- >> _______________________________________________ >> Openembedded-core mailing list >> Openembedded-core@lists.openembedded.org >> http://lists.openembedded.org/mailman/listinfo/openembedded-core [-- Attachment #2: Type: text/html, Size: 114418 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] python3: upgrade to 3.7.2 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego @ 2019-02-01 22:35 ` Alejandro Enedino Hernandez Samaniego 2019-02-03 2:08 ` Yu, Mingli 2019-02-04 15:01 ` Alexander Kanavin 2 siblings, 0 replies; 8+ messages in thread From: Alejandro Enedino Hernandez Samaniego @ 2019-02-01 22:35 UTC (permalink / raw) To: Khem Raj, Alexander Kanavin Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 120971 bytes --] On 2/1/19 2:32 PM, Alejandro Enedino Hernandez Samaniego wrote: > On 2/1/19 12:49 PM, Khem Raj wrote: >> On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin<alex.kanavin@gmail.com> wrote: >>> I took the same approach as the recent perl upgrade: write recipe from scratch, >>> taking the pieces from the old recipe only when they were proven to be necessary. >>> >>> The pgo, manifest and ptest features are all preserved. >>> >>> New features: >>> >>> - native and target recipes are now unified into one recipe >>> >>> - check_build_completeness.py runs right after do_compile() and verifies that >>> all optional modules have been built (a notorious source of regressions) >>> >>> - a new approach to sysconfig.py and distutils/sysconfig.py returning values >>> appropriate for native or target builds: we copy the configuration file to a >>> separate folder, add that folder to sys.path (through environment variable >>> that differs between native and target builds), and point python to the file >>> through another environment variable. >>> >> Thanks, once current cycle is over for OE, I would like to give this a >> whirl and see how it goes >> >>> Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> --- >>> meta/classes/python3-dir.bbclass | 2 +- >>> meta/classes/python3native.bbclass | 2 + >>> ...ib-termcap-to-linker-flags-to-avoid-.patch | 25 + >>> ...hell-version-of-python-config-that-w.patch | 35 + >>> ...-qemu-wrapper-when-gathering-profile.patch | 25 + >>> ...fig-append-STAGING_LIBDIR-python-sys.patch | 42 + >>> ...tutils-prefix-is-inside-staging-area.patch | 54 + >>> .../python3/avoid_warning_about_tkinter.patch | 36 + >>> .../python-sanity/python3/cgi_py.patch | 32 + >>> .../python3/check_build_completeness.py | 17 + >>> .../python-sanity/python3/create_manifest3.py | 433 ++++++ >>> .../python-sanity/python3/get_module_deps3.py | 146 ++ >>> .../python-sanity/python3/python-config.patch | 46 + >>> .../python3/python3-manifest.json | 1227 +++++++++++++++++ >>> .../python-sanity/python3/run-ptest | 3 + >>> .../python-sanity/python3_3.7.2.bb | 281 ++++ >>> 16 files changed, 2405 insertions(+), 1 deletion(-) >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest >>> create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb > Thanks Alex!, I looked at this briefly and it looks good. > Do we want to keep the recipe at recipes-devtools-python-sanity?, also I think the PACKAGECONFIG for tk is broken but thats probably tk's fault, > and, since tk is part of meta-oe only, I am wondering if it would make sense to have a bbappend there with the PACKAGECONFIG there, > since we're depending on something that were actually not holding? Actually nvm about moving the PACKAGECONFIG [tk] to meta-oe, I just remembered (1 minute too late) why we have it here and its because its necessary to build the complete python3-native package for the manifest, otherwise it wouldn't include the tkinter module, and we cant have like a different manifest in meta-oe, so it makes sense to keep it. Alejandro > Alejandro > > >>> diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass >>> index 06bb046d9c2..7dd130bad99 100644 >>> --- a/meta/classes/python3-dir.bbclass >>> +++ b/meta/classes/python3-dir.bbclass >>> @@ -1,4 +1,4 @@ >>> -PYTHON_BASEVERSION = "3.5" >>> +PYTHON_BASEVERSION = "3.7" >>> PYTHON_ABI = "m" >>> PYTHON_DIR = "python${PYTHON_BASEVERSION}" >>> PYTHON_PN = "python3" >>> diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass >>> index da12a714703..a3acaf61bbd 100644 >>> --- a/meta/classes/python3native.bbclass >>> +++ b/meta/classes/python3native.bbclass >>> @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native " >>> export STAGING_INCDIR >>> export STAGING_LIBDIR >>> >>> +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" >>> + >>> # suppress host user's site-packages dirs. >>> export PYTHONNOUSERSITE = "1" >>> >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> new file mode 100644 >>> index 00000000000..09f279ba1d7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> @@ -0,0 +1,25 @@ >>> +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Fri, 25 Jan 2019 19:04:13 +0100 >>> +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host >>> + contamination >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> + >>> +--- >>> + setup.py | 1 - >>> + 1 file changed, 1 deletion(-) >>> + >>> +diff --git a/setup.py b/setup.py >>> +index b4357e3..fbec00d 100644 >>> +--- a/setup.py >>> ++++ b/setup.py >>> +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext): >>> + 'termcap'): >>> + readline_libs.append('termcap') >>> + exts.append( Extension('readline', ['readline.c'], >>> +- library_dirs=['/usr/lib/termcap'], >>> + extra_link_args=readline_extra_link_args, >>> + libraries=readline_libs) ) >>> + else: >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> new file mode 100644 >>> index 00000000000..83fd52d87f4 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> @@ -0,0 +1,35 @@ >>> +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Tue, 29 Jan 2019 15:03:01 +0100 >>> +Subject: [PATCH] Do not use the shell version of python-config that was >>> + introduced in 3.4 >>> + >>> +Revert instead to the original python version: it has our tweaks and >>> +outputs directories correctly. >>> + >>> +Upstream-Status: Inappropriate [oe-specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> +--- >>> + Makefile.pre.in | 9 +++------ >>> + 1 file changed, 3 insertions(+), 6 deletions(-) >>> + >>> +diff --git a/Makefile.pre.in b/Makefile.pre.in >>> +index 2d2e11f..cc19942 100644 >>> +--- a/Makefile.pre.in >>> ++++ b/Makefile.pre.in >>> +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh >>> + sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py >>> + @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} >>> + LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config >>> +- @ # On Darwin, always use the python version of the script, the shell >>> +- @ # version doesn't use the compiler customizations that are provided >>> +- @ # in python (_osx_support.py). >>> +- @if test `uname -s` = Darwin; then \ >>> +- cp python-config.py python-config; \ >>> +- fi >>> ++ @ # In OpenEmbedded, always use the python version of the script, the shell >>> ++ @ # version is broken in multiple ways, and doesn't return correct directories >>> ++ cp python-config.py python-config >>> + >>> + >>> + # Install the include files >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> new file mode 100644 >>> index 00000000000..fa7735ff93e >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> @@ -0,0 +1,25 @@ >>> +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Wed, 30 Jan 2019 12:41:04 +0100 >>> +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> +--- >>> + Makefile.pre.in | 3 +-- >>> + 1 file changed, 1 insertion(+), 2 deletions(-) >>> + >>> +diff --git a/Makefile.pre.in b/Makefile.pre.in >>> +index a3a02a7..d5503dd 100644 >>> +--- a/Makefile.pre.in >>> ++++ b/Makefile.pre.in >>> +@@ -507,8 +507,7 @@ build_all_generate_profile: >>> + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" >>> + >>> + run_profile_task: >>> +- @ # FIXME: can't run for a cross build >>> +- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true >>> ++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true >>> + >>> + build_all_merge_profile: >>> + $(LLVM_PROF_MERGER) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> new file mode 100644 >>> index 00000000000..2c5b76f131c >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> @@ -0,0 +1,42 @@ >>> +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Thu, 31 Jan 2019 16:46:30 +0100 >>> +Subject: [PATCH] distutils/sysconfig: append >>> + STAGING_LIBDIR/python-sysconfigdata to sys.path >>> + >>> +So that target configuration can be used when running native python >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> + >>> +--- >>> + Lib/distutils/sysconfig.py | 2 ++ >>> + Lib/sysconfig.py | 2 ++ >>> + 2 files changed, 4 insertions(+) >>> + >>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >>> +index e07a6c8..6b8c129 100644 >>> +--- a/Lib/distutils/sysconfig.py >>> ++++ b/Lib/distutils/sysconfig.py >>> +@@ -421,6 +421,8 @@ def _init_posix(): >>> + platform=sys.platform, >>> + multiarch=getattr(sys.implementation, '_multiarch', ''), >>> + )) >>> ++ if 'STAGING_LIBDIR' in os.environ: >>> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >>> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >>> + build_time_vars = _temp.build_time_vars >>> + global _config_vars >>> +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py >>> +index 9ee4d31..e586abd 100644 >>> +--- a/Lib/sysconfig.py >>> ++++ b/Lib/sysconfig.py >>> +@@ -412,6 +412,8 @@ def _init_posix(vars): >>> + """Initialize the module as appropriate for POSIX systems.""" >>> + # _sysconfigdata is generated at build time, see _generate_posix_vars() >>> + name = _get_sysconfigdata_name() >>> ++ if 'STAGING_LIBDIR' in os.environ: >>> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >>> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >>> + build_time_vars = _temp.build_time_vars >>> + vars.update(build_time_vars) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> new file mode 100644 >>> index 00000000000..abf08f59dd7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> @@ -0,0 +1,54 @@ >>> +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001 >>> +From: Khem Raj<raj.khem@gmail.com> >>> +Date: Tue, 14 May 2013 15:00:26 -0700 >>> +Subject: [PATCH] python3: Add target and native recipes >>> + >>> +Upstream-Status: Inappropriate [embedded specific] >>> + >>> +02/2015 Rebased for Python 3.4.2 >>> + >>> +# The proper prefix is inside our staging area. >>> +# Signed-Off: Michael 'Mickey' Lauer<mickey@vanille-media.de> >>> +# Signed-off-by: Phil Blundell<philb@gnu.org> >>> +# Signed-off-by: Khem Raj<raj.khem@gmail.com> >>> +# Signed-off-by: Alejandro Hernandez<alejandro.hernandez@linux.intel.com> >>> + >>> +--- >>> + Lib/distutils/sysconfig.py | 10 ++++++++-- >>> + 1 file changed, 8 insertions(+), 2 deletions(-) >>> + >>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >>> +index 6b8c129..3ca7f79 100644 >>> +--- a/Lib/distutils/sysconfig.py >>> ++++ b/Lib/distutils/sysconfig.py >>> +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): >>> + If 'prefix' is supplied, use it instead of sys.base_prefix or >>> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >>> + """ >>> +- if prefix is None: >>> ++ if prefix is None and os.environ['STAGING_INCDIR'] != "": >>> ++ prefix = os.environ['STAGING_INCDIR'].rstrip('include') >>> ++ elif prefix is None: >>> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >>> + if os.name == "posix": >>> + if python_build: >>> +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >>> + If 'prefix' is supplied, use it instead of sys.base_prefix or >>> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >>> + """ >>> ++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] >>> ++ if prefix is None and os.environ['STAGING_LIBDIR'] != "": >>> ++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) >>> ++ >>> + if prefix is None: >>> + if standard_lib: >>> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >>> +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >>> + >>> + if os.name == "posix": >>> + libpython = os.path.join(prefix, >>> +- "lib", "python" + get_python_version()) >>> ++ lib_basename, "python" + get_python_version()) >>> + if standard_lib: >>> + return libpython >>> + else: >>> diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> new file mode 100644 >>> index 00000000000..24e67b4ca14 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> @@ -0,0 +1,36 @@ >>> +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001 >>> +From: Andrei Gherzan<andrei@gherzan.ro> >>> +Date: Mon, 28 Jan 2019 15:57:54 +0000 >>> +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet >>> + integrated in yocto so we skip the check for this module. Avoid a warning by >>> + not adding this module to missing variable. >>> + >>> +Upstream-Status: Inappropriate [distribution] >>> + >>> +Also simply disable the tk module since its not in DEPENDS. >>> +Signed-off-by: Andrei Gherzan<andrei@gherzan.ro> >>> + >>> +--- >>> + setup.py | 8 +++++--- >>> + 1 file changed, 5 insertions(+), 3 deletions(-) >>> + >>> +diff --git a/setup.py b/setup.py >>> +index fbec00d..b7a36a6 100644 >>> +--- a/setup.py >>> ++++ b/setup.py >>> +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext): >>> + self.extensions.extend(exts) >>> + >>> + # Call the method for detecting whether _tkinter can be compiled >>> +- self.detect_tkinter(inc_dirs, lib_dirs) >>> ++ # self.detect_tkinter(inc_dirs, lib_dirs) >>> + >>> +- if '_tkinter' not in [e.name for e in self.extensions]: >>> +- missing.append('_tkinter') >>> ++ # tkinter module will not be avalaible as yocto >>> ++ # doesn't have tk integrated (yet) >>> ++ #if '_tkinter' not in [e.name for e in self.extensions]: >>> ++ # missing.append('_tkinter') >>> + >>> + # Build the _uuid module if possible >>> + uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"]) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> new file mode 100644 >>> index 00000000000..6c4ba54320b >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> @@ -0,0 +1,32 @@ >>> +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001 >>> +From: Mark Hatle<mark.hatle@windriver.com> >>> +Date: Wed, 21 Sep 2011 20:55:33 -0500 >>> +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment >>> + >>> +Upstream-Status: Inappropriate [distribution] >>> + >>> +Signed-off-by: Mark Hatle<mark.hatle@windriver.com> >>> + >>> +--- >>> + Lib/cgi.py | 11 +---------- >>> + 1 file changed, 1 insertion(+), 10 deletions(-) >>> + >>> +diff --git a/Lib/cgi.py b/Lib/cgi.py >>> +index 8cf6687..094c7b4 100755 >>> +--- a/Lib/cgi.py >>> ++++ b/Lib/cgi.py >>> +@@ -1,13 +1,4 @@ >>> +-#! /usr/local/bin/python >>> +- >>> +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is >>> +-# intentionally NOT "/usr/bin/env python". On many systems >>> +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI >>> +-# scripts, and /usr/local/bin is the default directory where Python is >>> +-# installed, so /usr/bin/env would be unable to find python. Granted, >>> +-# binary installations by Linux vendors often install Python in >>> +-# /usr/bin. So let those vendors patch cgi.py to match their choice >>> +-# of installation. >>> ++#! /usr/bin/env python >>> + >>> + """Support module for CGI (Common Gateway Interface) scripts. >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> new file mode 100755 >>> index 00000000000..a1eace3f571 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> @@ -0,0 +1,17 @@ >>> +#!/usr/bin/env python3 >>> +import sys >>> +logfile = open(sys.argv[1]).read() >>> + >>> +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found") >>> +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.") >>> +if necessary_bits != -1: >>> + print("%s" %(logfile[necessary_bits:to_find_bits])) >>> + >>> +failed_to_build = logfile.find("Failed to build these modules:") >>> +if failed_to_build != -1: >>> + failed_to_build_end = logfile.find("\n\n", failed_to_build) >>> + print("%s" %(logfile[failed_to_build:failed_to_build_end])) >>> + >>> +if necessary_bits != -1 or failed_to_build != -1: >>> + sys.exit(1) >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> new file mode 100644 >>> index 00000000000..4da02a2991a >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> @@ -0,0 +1,433 @@ >>> +# This script is used as a bitbake task to create a new python manifest >>> +# $ bitbake python -c create_manifest >>> +# >>> +# Our goal is to keep python-core as small as posible and add other python >>> +# packages only when the user needs them, hence why we split upstream python >>> +# into several packages. >>> +# >>> +# In a very simplistic way what this does is: >>> +# Launch python and see specifically what is required for it to run at a minimum >>> +# >>> +# Go through the python-manifest file and launch a separate task for every single >>> +# one of the files on each package, this task will check what was required for that >>> +# specific module to run, these modules will be called dependencies. >>> +# The output of such task will be a list of the modules or dependencies that were >>> +# found for that file. >>> +# >>> +# Such output will be parsed by this script, we will look for each dependency on the >>> +# manifest and if we find that another package already includes it, then we will add >>> +# that package as an RDEPENDS to the package we are currently checking; in case we dont >>> +# find the current dependency on any other package we will add it to the current package >>> +# as part of FILES. >>> +# >>> +# >>> +# This way we will create a new manifest from the data structure that was built during >>> +# this process, on this new manifest each package will contain specifically only >>> +# what it needs to run. >>> +# >>> +# There are some caveats which we try to deal with, such as repeated files on different >>> +# packages, packages that include folders, wildcards, and special packages. >>> +# Its also important to note that this method only works for python files, and shared >>> +# libraries. Static libraries, header files and binaries need to be dealt with manually. >>> +# >>> +# This script differs from its python2 version mostly on how shared libraries are handled >>> +# The manifest file for python3 has an extra field which contains the cached files for >>> +# each package. >>> +# Tha method to handle cached files does not work when a module includes a folder which >>> +# itself contains the pycache folder, gladly this is almost never the case. >>> +# >>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> >>> + >>> + >>> +import sys >>> +import subprocess >>> +import json >>> +import os >>> +import collections >>> + >>> +# Get python version from ${PYTHON_MAJMIN} >>> +pyversion = str(sys.argv[1]) >>> + >>> +# Hack to get native python search path (for folders), not fond of it but it works for now >>> +pivot = 'recipe-sysroot-native' >>> +for p in sys.path: >>> + if pivot in p: >>> + nativelibfolder = p[:p.find(pivot)+len(pivot)] >>> + >>> +# Empty dict to hold the whole manifest >>> +new_manifest = collections.OrderedDict() >>> + >>> +# Check for repeated files, folders and wildcards >>> +allfiles = [] >>> +repeated = [] >>> +wildcards = [] >>> + >>> +hasfolders = [] >>> +allfolders = [] >>> + >>> +def isFolder(value): >>> + value = value.replace('${PYTHON_MAJMIN}',pyversion) >>> + if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): >>> + return True >>> + else: >>> + return False >>> + >>> +def isCached(item): >>> + if '__pycache__' in item: >>> + return True >>> + else: >>> + return False >>> + >>> +def prepend_comments(comments, json_manifest): >>> + with open(json_manifest, 'r+') as manifest: >>> + json_contents = manifest.read() >>> + manifest.seek(0, 0) >>> + manifest.write(comments + json_contents) >>> + >>> +# Read existing JSON manifest >>> +with open('python3-manifest.json') as manifest: >>> + # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker >>> + manifest_str = manifest.read() >>> + json_start = manifest_str.find('# EOC') + 6 # EOC + \n >>> + manifest.seek(0) >>> + comments = manifest.read(json_start) >>> + manifest_str = manifest.read() >>> + old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >>> + >>> +# >>> +# First pass to get core-package functionality, because we base everything on the fact that core is actually working >>> +# Not exactly the same so it should not be a function >>> +# >>> + >>> +print ('Getting dependencies for package: core') >>> + >>> + >>> +# This special call gets the core dependencies and >>> +# appends to the old manifest so it doesnt hurt what it >>> +# currently holds. >>> +# This way when other packages check for dependencies >>> +# on the new core package, they will still find them >>> +# even when checking the old_manifest >>> + >>> +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') >>> +for coredep in output.split(): >>> + coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + if isCached(coredep): >>> + if coredep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(coredep) >>> + else: >>> + if coredep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(coredep) >>> + >>> + >>> +# The second step is to loop through the existing files contained in the core package >>> +# according to the old manifest, identify if they are modules, or some other type >>> +# of file that we cant import (directories, binaries, configs) in which case we >>> +# can only assume they were added correctly (manually) so we ignore those and >>> +# pass them to the manifest directly. >>> + >>> +for filedep in old_manifest['core']['files']: >>> + if isFolder(filedep): >>> + if isCached(filedep): >>> + if filedep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(filedep) >>> + else: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + if '${bindir}' in filedep: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + if filedep == '': >>> + continue >>> + if '${includedir}' in filedep: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + >>> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >>> + pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] >>> + >>> + >>> + # We now know that were dealing with a python module, so we can import it >>> + # and check what its dependencies are. >>> + # We launch a separate task for each module for deterministic behavior. >>> + # Each module will only import what is necessary for it to work in specific. >>> + # The output of each task will contain each module's dependencies >>> + >>> + print ('Getting dependencies for module: %s' % pymodule) >>> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >>> + print ('The following dependencies were found for module %s:\n' % pymodule) >>> + print (output) >>> + >>> + >>> + for pymodule_dep in output.split(): >>> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + >>> + if isCached(pymodule_dep): >>> + if pymodule_dep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(pymodule_dep) >>> + >>> + >>> +# At this point we are done with the core package. >>> +# The old_manifest dictionary is updated only for the core package because >>> +# all others will use this a base. >>> + >>> + >>> +# To improve the script speed, we check which packages contain directories >>> +# since we will be looping through (only) those later. >>> +for pypkg in old_manifest: >>> + for filedep in old_manifest[pypkg]['files']: >>> + if isFolder(filedep): >>> + print ('%s is a folder' % filedep) >>> + if pypkg not in hasfolders: >>> + hasfolders.append(pypkg) >>> + if filedep not in allfolders: >>> + allfolders.append(filedep) >>> + >>> + >>> + >>> +# This is the main loop that will handle each package. >>> +# It works in a similar fashion than the step before, but >>> +# we will now be updating a new dictionary that will eventually >>> +# become the new manifest. >>> +# >>> +# The following loops though all packages in the manifest, >>> +# through all files on each of them, and checks whether or not >>> +# they are modules and can be imported. >>> +# If they can be imported, then it checks for dependencies for >>> +# each of them by launching a separate task. >>> +# The output of that task is then parsed and the manifest is updated >>> +# accordingly, wether it should add the module on FILES for the current package >>> +# or if that module already belongs to another package then the current one >>> +# will RDEPEND on it >>> + >>> +for pypkg in old_manifest: >>> + # Use an empty dict as data structure to hold data for each package and fill it up >>> + new_manifest[pypkg] = collections.OrderedDict() >>> + new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] >>> + new_manifest[pypkg]['rdepends'] = [] >>> + new_manifest[pypkg]['files'] = [] >>> + new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] >>> + >>> + # All packages should depend on core >>> + if pypkg != 'core': >>> + new_manifest[pypkg]['rdepends'].append('core') >>> + new_manifest[pypkg]['cached'] = [] >>> + >>> + print('\n') >>> + print('--------------------------') >>> + print ('Handling package %s' % pypkg) >>> + print('--------------------------') >>> + >>> + # Handle special cases, we assume that when they were manually added >>> + # to the manifest we knew what we were doing. >>> + special_packages = ['misc', 'modules', 'dev', 'tests'] >>> + if pypkg in special_packages or 'staticdev' in pypkg: >>> + print('Passing %s package directly' % pypkg) >>> + new_manifest[pypkg] = old_manifest[pypkg] >>> + continue >>> + >>> + for filedep in old_manifest[pypkg]['files']: >>> + # We already handled core on the first pass, we can ignore it now >>> + if pypkg == 'core': >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + >>> + # Handle/ignore what we cant import >>> + if isFolder(filedep): >>> + new_manifest[pypkg]['files'].append(filedep) >>> + # Asyncio (and others) are both the package and the folder name, we should not skip those... >>> + path,mod = os.path.split(filedep) >>> + if mod != pypkg: >>> + continue >>> + if '${bindir}' in filedep: >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + if filedep == '': >>> + continue >>> + if '${includedir}' in filedep: >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + >>> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >>> + # We need to check if the imported module comes from another (e.g. sqlite3.dump) >>> + path,pymodule = os.path.split(filedep) >>> + path = os.path.basename(path) >>> + pymodule = os.path.splitext(os.path.basename(pymodule))[0] >>> + >>> + # If this condition is met, it means we need to import it from another module >>> + # or its the folder itself (e.g. unittest) >>> + if path == pypkg: >>> + if pymodule: >>> + pymodule = path + '.' + pymodule >>> + else: >>> + pymodule = path >>> + >>> + >>> + >>> + # We now know that were dealing with a python module, so we can import it >>> + # and check what its dependencies are. >>> + # We launch a separate task for each module for deterministic behavior. >>> + # Each module will only import what is necessary for it to work in specific. >>> + # The output of each task will contain each module's dependencies >>> + >>> + print ('\nGetting dependencies for module: %s' % pymodule) >>> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >>> + print ('The following dependencies were found for module %s:\n' % pymodule) >>> + print (output) >>> + >>> + reportFILES = [] >>> + reportRDEPS = [] >>> + >>> + for pymodule_dep in output.split(): >>> + >>> + # Warning: This first part is ugly >>> + # One of the dependencies that was found, could be inside of one of the folders included by another package >>> + # We need to check if this happens so we can add the package containing the folder as an rdependency >>> + # e.g. Folder encodings contained in codecs >>> + # This would be solved if no packages included any folders >>> + >>> + # This can be done in two ways: >>> + # 1 - We assume that if we take out the filename from the path we would get >>> + # the folder string, then we would check if folder string is in the list of folders >>> + # This would not work if a package contains a folder which contains another folder >>> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >>> + # folder_string would not match any value contained in the list of folders >>> + # >>> + # 2 - We do it the other way around, checking if the folder is contained in the path >>> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >>> + # is folder_string inside path/folder1/folder2/filename?, >>> + # Yes, it works, but we waste a couple of milliseconds. >>> + >>> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + inFolders = False >>> + for folder in allfolders: >>> + # The module could have a directory named after it, e.g. xml, if we take out the filename from the path >>> + # we'll end up with ${libdir}, and we want ${libdir}/xml >>> + if isFolder(pymodule_dep): >>> + check_path = pymodule_dep >>> + else: >>> + check_path = os.path.dirname(pymodule_dep) >>> + if folder in check_path : >>> + inFolders = True # Did we find a folder? >>> + folderFound = False # Second flag to break inner for >>> + # Loop only through packages which contain folders >>> + for pypkg_with_folder in hasfolders: >>> + if (folderFound == False): >>> + # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) >>> + for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: >>> + if folder_dep == folder: >>> + print ('%s folder found in %s' % (folder, pypkg_with_folder)) >>> + folderFound = True >>> + if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: >>> + new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) >>> + else: >>> + break >>> + >>> + # A folder was found so we're done with this item, we can go on >>> + if inFolders: >>> + continue >>> + >>> + >>> + >>> + # No directories beyond this point >>> + # We might already have this module on the dictionary since it could depend on a (previously checked) module >>> + if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: >>> + # Handle core as a special package, we already did it so we pass it to NEW data structure directly >>> + if pypkg == 'core': >>> + print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) >>> + if pymodule_dep.endswith('*'): >>> + wildcards.append(pymodule_dep) >>> + if isCached(pymodule_dep): >>> + new_manifest[pypkg]['cached'].append(pymodule_dep) >>> + else: >>> + new_manifest[pypkg]['files'].append(pymodule_dep) >>> + >>> + # Check for repeated files >>> + if pymodule_dep not in allfiles: >>> + allfiles.append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in repeated: >>> + repeated.append(pymodule_dep) >>> + else: >>> + >>> + >>> + # Last step: Figure out if we this belongs to FILES or RDEPENDS >>> + # We check if this module is already contained on another package, so we add that one >>> + # as an RDEPENDS, or if its not, it means it should be contained on the current >>> + # package, and we should add it to FILES >>> + for possible_rdep in old_manifest: >>> + # Debug >>> + # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) >>> + if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: >>> + # Since were nesting, we need to check its not the same pypkg >>> + if(possible_rdep != pypkg): >>> + if possible_rdep not in new_manifest[pypkg]['rdepends']: >>> + # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs >>> + reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) >>> + new_manifest[pypkg]['rdepends'].append(possible_rdep) >>> + break >>> + else: >>> + >>> + # Since this module wasnt found on another package, it is not an RDEP, >>> + # so we add it to FILES for this package. >>> + # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) >>> + if os.path.basename(pymodule_dep) != pypkg: >>> + reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) >>> + if isCached(pymodule_dep): >>> + new_manifest[pypkg]['cached'].append(pymodule_dep) >>> + else: >>> + new_manifest[pypkg]['files'].append(pymodule_dep) >>> + if pymodule_dep.endswith('*'): >>> + wildcards.append(pymodule_dep) >>> + if pymodule_dep not in allfiles: >>> + allfiles.append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in repeated: >>> + repeated.append(pymodule_dep) >>> + >>> + print('\n') >>> + print('#################################') >>> + print('Summary for module %s' % pymodule) >>> + print('FILES found for module %s:' % pymodule) >>> + print(''.join(reportFILES)) >>> + print('RDEPENDS found for module %s:' % pymodule) >>> + print(''.join(reportRDEPS)) >>> + print('#################################') >>> + >>> +print('The following FILES contain wildcards, please check if they are necessary') >>> +print(wildcards) >>> +print('The following FILES contain folders, please check if they are necessary') >>> +print(hasfolders) >>> + >>> + >>> +# Sort it just so it looks nicer >>> +for pypkg in new_manifest: >>> + new_manifest[pypkg]['files'].sort() >>> + new_manifest[pypkg]['cached'].sort() >>> + new_manifest[pypkg]['rdepends'].sort() >>> + >>> +# Create the manifest from the data structure that was built >>> +with open('python3-manifest.json.new','w') as outfile: >>> + json.dump(new_manifest,outfile, indent=4) >>> + outfile.write('\n') >>> + >>> +prepend_comments(comments,'python3-manifest.json.new') >>> + >>> +if (repeated): >>> + error_msg = '\n\nERROR:\n' >>> + error_msg += 'The following files are repeated (contained in more than one package),\n' >>> + error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' >>> + error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' >>> + error_msg += '\n'.join(repeated) >>> + error_msg += '\n' >>> + sys.exit(error_msg) >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> new file mode 100644 >>> index 00000000000..fd12baad84e >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> @@ -0,0 +1,146 @@ >>> +# This script is launched on separate task for each python module >>> +# It checks for dependencies for that specific module and prints >>> +# them out, the output of this execution will have all dependencies >>> +# for a specific module, which will be parsed an dealt on create_manifest.py >>> +# >>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29"<aehs29@gmail.com> >>> + >>> +# We can get a log per module, for all the dependencies that were found, but its messy. >>> +debug=False >>> + >>> +import sys >>> + >>> +# We can get a list of the modules which are currently required to run python >>> +# so we run python-core and get its modules, we then import what we need >>> +# and check what modules are currently running, if we substract them from the >>> +# modules we had initially, we get the dependencies for the module we imported. >>> + >>> +# We use importlib to achieve this, so we also need to know what modules importlib needs >>> +import importlib >>> + >>> +core_deps=set(sys.modules) >>> + >>> +def fix_path(dep_path): >>> + import os >>> + # We DONT want the path on our HOST system >>> + pivot='recipe-sysroot-native' >>> + dep_path=dep_path[dep_path.find(pivot)+len(pivot):] >>> + >>> + if '/usr/bin' in dep_path: >>> + dep_path = dep_path.replace('/usr/bin''${bindir}') >>> + >>> + # Handle multilib, is there a better way? >>> + if '/usr/lib32' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib32','${libdir}') >>> + if '/usr/lib64' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib64','${libdir}') >>> + if '/usr/lib' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib','${libdir}') >>> + if '/usr/include' in dep_path: >>> + dep_path = dep_path.replace('/usr/include','${includedir}') >>> + if '__init__.' in dep_path: >>> + dep_path = os.path.split(dep_path)[0] >>> + return dep_path >>> + >>> + >>> +# Module to import was passed as an argument >>> +current_module = str(sys.argv[1]).rstrip() >>> +if(debug==True): >>> + log = open('log_%s' % current_module,'w') >>> + log.write('Module %s generated the following dependencies:\n' % current_module) >>> +try: >>> + importlib.import_module('%s' % current_module) >>> +except ImportError as e: >>> + if (debug==True): >>> + log.write('Module was not found') >>> + pass >>> + >>> + >>> +# Get current module dependencies, dif will contain a list of specific deps for this module >>> +module_deps=set(sys.modules) >>> + >>> +# We handle the core package (1st pass on create_manifest.py) as a special case >>> +if current_module == 'python-core-package': >>> + dif = core_deps >>> +else: >>> + # We know this is not the core package, so there must be a difference. >>> + dif = module_deps-core_deps >>> + >>> + >>> +# Check where each dependency came from >>> +for item in dif: >>> + dep_path='' >>> + try: >>> + if (debug==True): >>> + log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') >>> + dep_path = sys.modules['%s' % item].__file__ >>> + except AttributeError as e: >>> + # Deals with thread (builtin module) not having __file__ attribute >>> + if debug==True: >>> + log.write(item + ' ') >>> + log.write(str(e)) >>> + log.write('\n') >>> + pass >>> + except NameError as e: >>> + # Deals with NameError: name 'dep_path' is not defined >>> + # because module is not found (wasn't compiled?), e.g. bddsm >>> + if (debug==True): >>> + log.write(item+' ') >>> + log.write(str(e)) >>> + pass >>> + >>> + # Site-customize is a special case since we (OpenEmbedded) put it there manually >>> + if 'sitecustomize' in dep_path: >>> + dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' >>> + # Prints out result, which is what will be used by create_manifest >>> + print (dep_path) >>> + continue >>> + >>> + dep_path = fix_path(dep_path) >>> + >>> + import sysconfig >>> + soabi=sysconfig.get_config_var('SOABI') >>> + # Check if its a shared library and deconstruct it >>> + if soabi in dep_path: >>> + if (debug==True): >>> + log.write('Shared library found in %s' % dep_path) >>> + dep_path = dep_path.replace(soabi,'*') >>> + print (dep_path) >>> + continue >>> + >>> + if (debug==True): >>> + log.write(dep_path+'\n') >>> + # Prints out result, which is what will be used by create_manifest >>> + print (dep_path) >>> + >>> + >>> + import imp >>> + cpython_tag = imp.get_tag() >>> + cached='' >>> + # Theres no naive way to find *.pyc files on python3 >>> + try: >>> + if (debug==True): >>> + log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') >>> + cached = sys.modules['%s' % item].__cached__ >>> + except AttributeError as e: >>> + # Deals with thread (builtin module) not having __cached__ attribute >>> + if debug==True: >>> + log.write(item + ' ') >>> + log.write(str(e)) >>> + log.write('\n') >>> + pass >>> + except NameError as e: >>> + # Deals with NameError: name 'cached' is not defined >>> + if (debug==True): >>> + log.write(item+' ') >>> + log.write(str(e)) >>> + pass >>> + if cached is not None: >>> + if (debug==True): >>> + log.write(cached) >>> + cached = fix_path(cached) >>> + cached = cached.replace(cpython_tag,'*') >>> + print (cached) >>> + >>> +if debug==True: >>> + log.close() >>> diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch >>> new file mode 100644 >>> index 00000000000..f23b8b7df06 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch >>> @@ -0,0 +1,46 @@ >>> +python-config: Revert to using distutils.sysconfig >>> + >>> +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in >>> + >>> +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig >>> + >>> +affect the native runtime as well as cross building. Use the old, patched >>> +implementation which returns paths in the staging directory and for the target, >>> +as appropriate. >>> + >>> +Upstream-Status: Inappropriate [Embedded Specific] >>> + >>> +Signed-off-by: Tyler Hall<tylerwhall@gmail.com> >>> +: >>> +Index: Python-3.3.3/Misc/python-config.in >>> +=================================================================== >>> +--- Python-3.3.3.orig/Misc/python-config.in >>> ++++ Python-3.3.3/Misc/python-config.in >>> +@@ -4,7 +4,7 @@ >>> + import getopt >>> + import os >>> + import sys >>> +-import sysconfig >>> ++from distutils import sysconfig >>> + >>> + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', >>> + 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] >>> +@@ -32,14 +32,14 @@ if '--help' in opt_flags: >>> + >>> + for opt in opt_flags: >>> + if opt == '--prefix': >>> +- print(sysconfig.get_config_var('prefix')) >>> ++ print(sysconfig.PREFIX) >>> + >>> + elif opt == '--exec-prefix': >>> +- print(sysconfig.get_config_var('exec_prefix')) >>> ++ print(sysconfig.EXEC_PREFIX) >>> + >>> + elif opt in ('--includes', '--cflags'): >>> +- flags = ['-I' + sysconfig.get_path('include'), >>> +- '-I' + sysconfig.get_path('platinclude')] >>> ++ flags = ['-I' + sysconfig.get_python_inc(), >>> ++ '-I' + sysconfig.get_python_inc(plat_specific=True)] >>> + if opt == '--cflags': >>> + flags.extend(getvar('CFLAGS').split()) >>> + print(' '.join(flags)) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> new file mode 100644 >>> index 00000000000..82c7075f0e7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> @@ -0,0 +1,1227 @@ >>> +# DO NOT (entirely) modify this file manually, please read. >>> +# >>> +# IMPORTANT NOTE: >>> +# Please keep in mind that the create_manifest task relies on the fact the the >>> +# target and native Python packages are the same, and it also needs to be executed >>> +# with a fully working native package (with all the PACKAGECONFIGs enabled and all >>> +# and all the modules should be working, check log.do_compile), otherwise the script >>> +# will fail to find dependencies correctly, this note is valid either if you are >>> +# upgrading to a new Python version or adding a new package. >>> +# >>> +# >>> +# If you are adding a new package please follow the next steps: >>> +# How to add a new package: >>> +# - If a user wants to add a new package all that has to be done is: >>> +# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, >>> +# fill up the SUMMARY section as well, the script should handle all the rest. >>> +# >>> +# Real example: >>> +# We want to add a web browser package, including the file webbrowser.py >>> +# which at the moment is on python3-misc. >>> +# "webbrowser": { >>> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], >>> +# "rdepends": [], >>> +# "summary": "Python Web Browser support"} >>> +# >>> +# * Note that the rdepends field was left empty >>> +# >>> +# We run $ bitbake python3 -c create_manifest and the resulting manifest >>> +# should be completed after a few seconds, showing something like: >>> +# "webbrowser": { >>> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], >>> +# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], >>> +# "summary": "Python Web Browser support"} >>> +# >>> +# >>> +# If you are upgrading Python to a new version please follow the next steps: >>> +# After each Python upgrade, the create_manifest task should be executed, because we >>> +# don't control what changes on upstream Python, so, some module dependency >>> +# might have changed without us realizing it, a certain module can either have >>> +# more or less dependencies, or could be depending on a new file that was just >>> +# created on the new release and for obvious reasons we wouldn't have it on our >>> +# old manifest, all of these issues would cause runtime errors on our system. >>> +# >>> +# - Upgrade both the native and target Python packages to a new version >>> +# - Run the create_manifest task for the target Python package as its shown below: >>> +# >>> +# $ bitbake python3 -c create_manifest >>> +# >>> +# This will automatically replace your manifest file located under the Python directory >>> +# with an new one, which contains the new dependencies (if any). >>> +# >>> +# Several things could have gone wrong here, I will try to explain a few: >>> +# >>> +# a) A new file was introduced on this release, e.g. sha3*.so: >>> +# The task will check what its needed to import every module, more than one module would >>> +# would probably depend on sha3*.so, although only one module should contain it. >>> +# >>> +# After running the task, the new manifest will have the sha3*.so file on more than one >>> +# module, you need to manually decide which one of them should get it and delete it from >>> +# the others, for example sha3*.so should likely be on ${PN}-crypt. >>> +# Once you have deleted from the others you need to run the create_manifest task again, >>> +# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be >>> +# good to go. >>> +# >>> +# b) The native package wasn't built correctly and its missing a certain module: >>> +# As mentioned before, you need to make sure the native package was built with all the modules >>> +# because it is used as base to build the manifest file, you need to manually check log.do_compile >>> +# since it won't error out the compile function if its only missing a couple of modules. >>> +# >>> +# e.g. missing the _uuid module, log.do_compile would show the following: >>> +# Python build finished successfully! >>> +# The necessary bits to build these optional modules were not found: >>> +# _uuid >>> +# >>> +# What will happen here is that the new manifest would not be aware that the _uuid module exists, so >>> +# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on >>> +# the misc package (which is where any file that doesn't belong anywhere else ends up). >>> +# >>> +# This will eventually cause runtime errors on our system if we don't include the misc package on >>> +# on our image, because the _uuid files will be missing. >>> +# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be >>> +# detected correctly along with its dependencies, and we will get a working manifest. >>> +# >>> +# This is the reason why it is important to make sure we have a fully working native build, >>> +# so we can avoid these errors. >>> +# >>> +# >>> +# >>> +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST >>> +# EOC >>> +{ >>> + "tests": { >>> + "summary": "Python test suite", >>> + "rdepends": [ >>> + "core", >>> + "modules" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/*/test", >>> + "${libdir}/python${PYTHON_MAJMIN}/*/tests", >>> + "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", >>> + "${libdir}/python${PYTHON_MAJMIN}/test" >>> + ], >>> + "cached": [] >>> + }, >>> + "2to3": { >>> + "summary": "Python automated Python 2 to 3 code translator", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${bindir}/2to3-*", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib2to3" >>> + ], >>> + "cached": [] >>> + }, >>> + "asyncio": { >>> + "summary": "Python Asynchronous I/", >>> + "rdepends": [ >>> + "core", >>> + "io", >>> + "logging", >>> + "netclient", >>> + "numbers", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/asyncio", >>> + "${libdir}/python${PYTHON_MAJMIN}/concurrent", >>> + "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "audio": { >>> + "summary": "Python Audio Handling", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/chunk.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sunau.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/wave.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" >>> + ] >>> + }, >>> + "codecs": { >>> + "summary": "Python codec", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" >>> + ] >>> + }, >>> + "compile": { >>> + "summary": "Python bytecode compilation support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/compileall.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" >>> + ] >>> + }, >>> + "compression": { >>> + "summary": "Python high-level compression support", >>> + "rdepends": [ >>> + "core", >>> + "shell", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_compression.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/bz2.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/gzip.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lzma.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" >>> + ] >>> + }, >>> + "core": { >>> + "summary": "Python interpreter and core modules", >>> + "rdepends": [], >>> + "files": [ >>> + "${bindir}/python*[!-config]", >>> + "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserList.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserString.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/__future__.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/argparse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ast.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/bisect.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/code.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/codecs.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/codeop.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/configparser.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/copy.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/csv.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/dis.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/enum.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/functools.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/getopt.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/gettext.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/heapq.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/imp.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/inspect.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/io.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/keyword.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/linecache.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/locale.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/new.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/opcode.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/operator.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/optparse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/os.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/platform.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/re.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/selectors.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/signal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/site.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/stat.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/struct.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/symbol.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/threading.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/token.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/traceback.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/types.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/warnings.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/weakref.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" >>> + ] >>> + }, >>> + "crypt": { >>> + "summary": "Python basic cryptographic and hashing support", >>> + "rdepends": [ >>> + "core", >>> + "math", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/crypt.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" >>> + ] >>> + }, >>> + "ctypes": { >>> + "summary": "Python C types support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/ctypes", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "curses": { >>> + "summary": "Python curses support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/curses", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "datetime": { >>> + "summary": "Python calendar and time support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/calendar.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/datetime.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" >>> + ] >>> + }, >>> + "db": { >>> + "summary": "Python file-based database support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/dbm", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "debugger": { >>> + "summary": "Python debugger", >>> + "rdepends": [ >>> + "core", >>> + "pprint", >>> + "shell", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/bdb.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pdb.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" >>> + ] >>> + }, >>> + "dev": { >>> + "cached": [], >>> + "files": [ >>> + "${base_libdir}/*.a", >>> + "${base_libdir}/*.o", >>> + "${bindir}/python*-config", >>> + "${datadir}/aclocal", >>> + "${datadir}/pkgconfig", >>> + "${includedir}", >>> + "${libdir}/*.a", >>> + "${libdir}/*.la", >>> + "${libdir}/*.o", >>> + "${libdir}/lib*${SOLIBSDEV}", >>> + "${libdir}/pkgconfig", >>> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", >>> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" >>> + ], >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "summary": "Python development package" >>> + }, >>> + "difflib": { >>> + "summary": "Python helpers for computing deltas between objects", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/difflib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" >>> + ] >>> + }, >>> + "distutils-staticdev": { >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" >>> + ], >>> + "rdepends": [ >>> + "distutils" >>> + ], >>> + "summary": "Python distribution utilities (static libraries)" >>> + }, >>> + "distutils": { >>> + "summary": "Python Distribution Utilities", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/distutils" >>> + ], >>> + "cached": [] >>> + }, >>> + "doctest": { >>> + "summary": "Python framework for running examples in docstrings", >>> + "rdepends": [ >>> + "core", >>> + "debugger", >>> + "difflib", >>> + "logging", >>> + "pprint", >>> + "shell", >>> + "stringold", >>> + "unittest" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/doctest.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" >>> + ] >>> + }, >>> + "email": { >>> + "summary": "Python email support", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "io", >>> + "math", >>> + "netclient" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/email", >>> + "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" >>> + ] >>> + }, >>> + "fcntl": { >>> + "summary": "Python's fcntl interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "gdbm": { >>> + "summary": "Python GNU database support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "html": { >>> + "summary": "Python HTML processing support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/formatter.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/html" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" >>> + ] >>> + }, >>> + "idle": { >>> + "summary": "Python Integrated Development Environment", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${bindir}/idle*", >>> + "${libdir}/python${PYTHON_MAJMIN}/idlelib" >>> + ], >>> + "cached": [] >>> + }, >>> + "image": { >>> + "summary": "Python graphical image handling", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" >>> + ] >>> + }, >>> + "io": { >>> + "summary": "Python low-level I/O", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "crypt", >>> + "math", >>> + "netclient", >>> + "shell", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/pipes.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/socket.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ssl.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" >>> + ] >>> + }, >>> + "json": { >>> + "summary": "Python JSON support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/json", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "logging": { >>> + "summary": "Python logging support", >>> + "rdepends": [ >>> + "core", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/logging" >>> + ], >>> + "cached": [] >>> + }, >>> + "mailbox": { >>> + "summary": "Python mailbox format support", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "fcntl", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" >>> + ] >>> + }, >>> + "math": { >>> + "summary": "Python math support", >>> + "rdepends": [ >>> + "core", >>> + "crypt" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/random.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" >>> + ] >>> + }, >>> + "mime": { >>> + "summary": "Python MIME handling APIs", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/quopri.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/uu.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" >>> + ] >>> + }, >>> + "mmap": { >>> + "summary": "Python memory-mapped file support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "modules": { >>> + "cached": [], >>> + "files": [], >>> + "rdepends": [ >>> + "2to3", >>> + "asyncio", >>> + "audio", >>> + "codecs", >>> + "compile", >>> + "compression", >>> + "core", >>> + "crypt", >>> + "ctypes", >>> + "curses", >>> + "datetime", >>> + "db", >>> + "debugger", >>> + "difflib", >>> + "distutils", >>> + "doctest", >>> + "email", >>> + "fcntl", >>> + "html", >>> + "idle", >>> + "image", >>> + "io", >>> + "json", >>> + "logging", >>> + "mailbox", >>> + "math", >>> + "mime", >>> + "mmap", >>> + "multiprocessing", >>> + "netclient", >>> + "netserver", >>> + "numbers", >>> + "pickle", >>> + "pkgutil", >>> + "plistlib", >>> + "pprint", >>> + "profile", >>> + "pydoc", >>> + "resource", >>> + "runpy", >>> + "shell", >>> + "smtpd", >>> + "sqlite3", >>> + "stringold", >>> + "syslog", >>> + "terminal", >>> + "threading", >>> + "tkinter", >>> + "typing", >>> + "unittest", >>> + "unixadmin", >>> + "venv", >>> + "xml", >>> + "xmlrpc" >>> + ], >>> + "summary": "All Python modules" >>> + }, >>> + "multiprocessing": { >>> + "summary": "Python multiprocessing support", >>> + "rdepends": [ >>> + "core", >>> + "io", >>> + "pickle" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" >>> + ], >>> + "cached": [] >>> + }, >>> + "netclient": { >>> + "summary": "Python Internet Protocol clients", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "io", >>> + "math", >>> + "mime", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/base64.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/hmac.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/http", >>> + "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/poplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/urllib", >>> + "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/uuid.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" >>> + ] >>> + }, >>> + "netserver": { >>> + "summary": "Python Internet Protocol servers", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "html", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "shell", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cgi.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" >>> + ] >>> + }, >>> + "numbers": { >>> + "summary": "Python number APIs", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/contextvars.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/decimal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/fractions.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/numbers.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" >>> + ] >>> + }, >>> + "pickle": { >>> + "summary": "Python serialisation/persistence support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/pickle.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shelve.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" >>> + ] >>> + }, >>> + "pkgutil": { >>> + "summary": "Python package extension utility support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" >>> + ] >>> + }, >>> + "plistlib": { >>> + "summary": "Generate and parse Mac OS X .plist files", >>> + "rdepends": [ >>> + "core", >>> + "datetime", >>> + "xml" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" >>> + ] >>> + }, >>> + "pprint": { >>> + "summary": "Python pretty-print support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pprint.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" >>> + ] >>> + }, >>> + "profile": { >>> + "summary": "Python basic performance profiling support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/profile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pstats.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" >>> + ] >>> + }, >>> + "pydoc": { >>> + "summary": "Python interactive help support", >>> + "rdepends": [ >>> + "core", >>> + "netclient", >>> + "pkgutil" >>> + ], >>> + "files": [ >>> + "${bindir}/pydoc*", >>> + "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" >>> + ] >>> + }, >>> + "resource": { >>> + "summary": "Python resource control interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "runpy": { >>> + "summary": "Python helper for locating/executing scripts in module namespace", >>> + "rdepends": [ >>> + "core", >>> + "pkgutil" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/runpy.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" >>> + ] >>> + }, >>> + "shell": { >>> + "summary": "Python shell-like functionality", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cmd.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/glob.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shlex.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shutil.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" >>> + ] >>> + }, >>> + "smtpd": { >>> + "summary": "Python Simple Mail Transport Daemon", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${bindir}/smtpd.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" >>> + ] >>> + }, >>> + "sqlite3": { >>> + "summary": "Python Sqlite3 database support", >>> + "rdepends": [ >>> + "core", >>> + "datetime" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/sqlite3" >>> + ], >>> + "cached": [] >>> + }, >>> + "stringold": { >>> + "summary": "Python string APIs [deprecated]", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/string.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" >>> + ] >>> + }, >>> + "syslog": { >>> + "summary": "Python syslog interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "terminal": { >>> + "summary": "Python terminal controlling support", >>> + "rdepends": [ >>> + "core", >>> + "io" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pty.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tty.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" >>> + ] >>> + }, >>> + "threading": { >>> + "summary": "Python threading & synchronization support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/queue.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" >>> + ] >>> + }, >>> + "tkinter": { >>> + "summary": "Python Tcl/Tk bindings", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/tkinter" >>> + ], >>> + "cached": [] >>> + }, >>> + "typing": { >>> + "summary": "Python typing support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/typing.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" >>> + ] >>> + }, >>> + "unittest": { >>> + "summary": "Python unit testing framework", >>> + "rdepends": [ >>> + "core", >>> + "difflib", >>> + "logging", >>> + "pprint", >>> + "shell", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest", >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest/", >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" >>> + ], >>> + "cached": [] >>> + }, >>> + "unixadmin": { >>> + "summary": "Python Unix administration support", >>> + "rdepends": [ >>> + "core", >>> + "io" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/getpass.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" >>> + ] >>> + }, >>> + "venv": { >>> + "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "logging", >>> + "shell", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${bindir}/pyvenv*", >>> + "${libdir}/python${PYTHON_MAJMIN}/venv" >>> + ], >>> + "cached": [] >>> + }, >>> + "xml": { >>> + "summary": "Python basic XML support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/xml" >>> + ], >>> + "cached": [] >>> + }, >>> + "xmlrpc": { >>> + "summary": "Python XML-RPC support", >>> + "rdepends": [ >>> + "core", >>> + "xml" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", >>> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" >>> + ], >>> + "cached": [] >>> + } >>> +} >>> diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest >>> new file mode 100644 >>> index 00000000000..3863c6d314f >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest >>> @@ -0,0 +1,3 @@ >>> +#!/bin/sh >>> + >>> +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' >>> diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >>> new file mode 100644 >>> index 00000000000..61fa12ffe8d >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >>> @@ -0,0 +1,281 @@ >>> +SUMMARY = "The Python Programming Language" >>> +HOMEPAGE ="http://www.python.org" >>> +LICENSE = "PSFv2" >>> +SECTION = "devel/python" >>> + >>> +LIC_FILES_CHKSUM ="file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754" >>> + >>> +SRC_URI ="http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ + >>> file://run-ptest \ + file://create_manifest3.py \ + >>> file://get_module_deps3.py \ + file://python3-manifest.json \ + >>> file://check_build_completeness.py \ + file://cgi_py.patch \ + >>> file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> \ + ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', >>> 'file://avoid_warning_about_tkinter.patch', d)} \ + >>> file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> \ + file://python-config.patch \ + >>> file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> \ + " >>> + >>> +SRC_URI_append_class-native = " \ >>> +file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ >>> +file://12-distutils-prefix-is-inside-staging-area.patch \ >>> + " >>> + >>> +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb" >>> +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb" >>> + >>> +# exclude pre-releases for both python 2.x and 3.x >>> +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" >>> + >>> +CVE_PRODUCT = "python" >>> + >>> +PYTHON_MAJMIN = "3.7" >>> +PYTHON_BINABI = "${PYTHON_MAJMIN}m" >>> + >>> +S = "${WORKDIR}/Python-${PV}" >>> + >>> +BBCLASSEXTEND = "native nativesdk" >>> + >>> +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives >>> + >>> +MULTILIB_SUFFIX ="${@d.getVar('base_libdir',1).split('/')[-1]}" >>> + >>> +ALTERNATIVE_${PN}-dev = "python-config" >>> +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" >>> +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" >>> + >>> + >>> +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2" >>> +DEPENDS_append_class-target = " python3-native" >>> +DEPENDS_append_class-nativesdk = " python3-native" >>> + >>> +EXTRA_OECONF = " --without-ensurepip --enable-shared" >>> +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}" >>> + >>> + >>> +EXTRANATIVEPATH += "python3-native" >>> + >>> +CACHED_CONFIGUREVARS = " \ >>> + ac_cv_file__dev_ptmx=yes \ >>> + ac_cv_file__dev_ptc=no \ >>> +" >>> + >>> +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" >>> +PACKAGECONFIG_class-native ??= "readline" >>> +PACKAGECONFIG_class-nativesdk ??= "readline" >>> +PACKAGECONFIG[readline] = ",,readline" >>> +# Use profile guided optimisation by running PyBench inside qemu-user >>> +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native" >>> +PACKAGECONFIG[tk] = ",,tk" >>> + >>> +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid" >>> + >>> +EXTRA_OEMAKE = '\ >>> + STAGING_LIBDIR=${STAGING_LIBDIR} \ >>> + STAGING_INCDIR=${STAGING_INCDIR} \ >>> +' >>> + >>> +do_compile_prepend_class-target() { >>> + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then >>> + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" >>> + cat >pgo-wrapper <<EOF >>> +#!/bin/sh >>> +cd ${B} >>> +$qemu_binary "\$@" >>> +EOF >>> + chmod +x pgo-wrapper >>> + fi >>> +} >>> + >>> +do_install_prepend() { >>> + ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile >>> +} >>> + >>> +do_install_append_class-target() { >>> + oe_multilib_header python${PYTHON_BINABI}/pyconfig.h >>> +} >>> + >>> +do_install_append_class-native() { >>> + # Make sure we use /usr/bin/env python >>> + for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do >>> + sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT >>> + done >>> + # Add a symlink to the native Python so that scripts can just invoke >>> + # "nativepython" and get the right one without needing absolute paths >>> + # (these often end up too long for the #! parser in the kernel as the >>> + # buffer is 128 bytes long). >>> + ln -s python3-native/python3 ${D}${bindir}/nativepython3 >>> +} >>> + >>> +do_install_append() { >>> + mkdir -p ${D}${libdir}/python-sysconfigdata >>> + sysconfigfile=`find ${D} -name _sysconfig*.py` >>> + cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >>> + >>> + sed -i \ >>> + -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \ >>> + -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >>> + -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >>> + ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >>> +} >>> + >>> +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py" >>> +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" >>> + >>> +py_package_preprocess () { >>> + # Remove references to buildmachine paths in target Makefile and _sysconfigdata >>> + sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ >>> + -e 's|${DEBUG_PREFIX_MAP}||g' \ >>> + -e 's:${HOSTTOOLS_DIR}/::g' \ >>> + -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ >>> + -e 's:${RECIPE_SYSROOT}::g' \ >>> + -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ >>> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \ >>> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \ >>> + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config >>> + >>> + # Recompile _sysconfigdata after modifying it >>> + cd ${PKGD} >>> + sysconfigfile=`find . -name _sysconfigdata_*.py` >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile')" >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)" >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)" >>> + cd - >>> + >>> + mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} >>> + >>> + #Remove the unneeded copy of target sysconfig data >>> + rm -rf ${PKGD}/${libdir}/python-sysconfigdata >>> +} >>> + >>> +# We want bytecode precompiled .py files (.pyc's) by default >>> +# but the user may set it on their own conf >>> +INCLUDE_PYCS ?= "1" >>> + >>> +python(){ >>> + import collections, json >>> + >>> + filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') >>> + # This python changes the datastore based on the contents of a file, so mark >>> + # that dependency. >>> + bb.parse.mark_dependency(d, filename) >>> + >>> + with open(filename) as manifest_file: >>> + manifest_str = manifest_file.read() >>> + json_start = manifest_str.find('# EOC') + 6 >>> + manifest_file.seek(json_start) >>> + manifest_str = manifest_file.read() >>> + python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >>> + >>> + # First set RPROVIDES for -native case >>> + # Hardcoded since it cant be python3-native-foo, should be python3-foo-native >>> + pn = 'python3' >>> + rprovides = d.getVar('RPROVIDES').split() >>> + >>> + for key in python_manifest: >>> + pypackage = pn + '-' + key + '-native' >>> + if pypackage not in rprovides: >>> + rprovides.append(pypackage) >>> + >>> + d.setVar('RPROVIDES_class-native', ' '.join(rprovides)) >>> + >>> + # Then work on the target >>> + include_pycs = d.getVar('INCLUDE_PYCS') >>> + >>> + packages = d.getVar('PACKAGES').split() >>> + pn = d.getVar('PN') >>> + >>> + newpackages=[] >>> + for key in python_manifest: >>> + pypackage= pn + '-' + key >>> + >>> + if pypackage not in packages: >>> + # We need to prepend, otherwise python-misc gets everything >>> + # so we use a new variable >>> + newpackages.append(pypackage) >>> + >>> + # "Build" python's manifest FILES, RDEPENDS and SUMMARY >>> + d.setVar('FILES_' + pypackage, '') >>> + for value in python_manifest[key]['files']: >>> + d.appendVar('FILES_' + pypackage, ' ' + value) >>> + >>> + # Add cached files >>> + if include_pycs == '1': >>> + for value in python_manifest[key]['cached']: >>> + d.appendVar('FILES_' + pypackage, ' ' + value) >>> + >>> + for value in python_manifest[key]['rdepends']: >>> + # Make it work with or without $PN >>> + if '${PN}' in value: >>> + value=value.split('-')[1] >>> + d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) >>> + d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) >>> + >>> + # Prepending so to avoid python-misc getting everything >>> + packages = newpackages + packages >>> + d.setVar('PACKAGES', ' '.join(packages)) >>> + d.setVar('ALLOW_EMPTY_${PN}-modules', '1') >>> +} >>> + >>> +# Files needed to create a new manifest >>> + >>> +do_create_manifest() { >>> + # This task should be run with every new release of Python. >>> + # We must ensure that PACKAGECONFIG enables everything when creating >>> + # a new manifest, this is to base our new manifest on a complete >>> + # native python build, containing all dependencies, otherwise the task >>> + # wont be able to find the required files. >>> + # e.g. BerkeleyDB is an optional build dependency so it may or may not >>> + # be present, we must ensure it is. >>> + >>> + cd ${WORKDIR} >>> + # This needs to be executed by python-native and NOT by HOST's python >>> + nativepython3 create_manifest3.py ${PYTHON_MAJMIN} >>> + cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json >>> +} >>> + >>> +# bitbake python -c create_manifest >>> +addtask do_create_manifest >>> + >>> +# Make sure we have native python ready when we create a new manifest >>> +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" >>> +do_create_manifest[depends] += "python3:do_patch" >>> + >>> +# manual dependency additions >>> +RPROVIDES_${PN}-modules = "${PN}" >>> +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" >>> +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" >>> + >>> +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" >>> +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" >>> +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" >>> + >>> +# provide python-pyvenv from python3-venv >>> +RPROVIDES_${PN}-venv += "python3-pyvenv" >>> + >>> +# package libpython3 >>> +PACKAGES =+ "libpython3 libpython3-staticdev" >>> +FILES_libpython3 = "${libdir}/libpython*.so.*" >>> +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a" >>> +INSANE_SKIP_${PN}-dev += "dev-elf" >>> + >>> +# catch all the rest (unsorted) >>> +PACKAGES += "${PN}-misc" >>> +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs" >>> +RDEPENDS_${PN}-modules += "python3-misc" >>> +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" >>> + >>> +# catch manpage >>> +PACKAGES += "${PN}-man" >>> +FILES_${PN}-man = "${datadir}/man" >>> + >>> +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" >>> +RDEPENDS_${PN}-tkinter +="${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" >>> +RDEPENDS_${PN}-dev = "" >>> + >>> -- >>> 2.17.1 >>> >>> -- >>> _______________________________________________ >>> Openembedded-core mailing list >>> Openembedded-core@lists.openembedded.org >>> http://lists.openembedded.org/mailman/listinfo/openembedded-core [-- Attachment #2: Type: text/html, Size: 115992 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] python3: upgrade to 3.7.2 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego 2019-02-01 22:35 ` Alejandro Enedino Hernandez Samaniego @ 2019-02-03 2:08 ` Yu, Mingli 2019-02-04 15:01 ` Alexander Kanavin 2 siblings, 0 replies; 8+ messages in thread From: Yu, Mingli @ 2019-02-03 2:08 UTC (permalink / raw) To: Alejandro Enedino Hernandez Samaniego, Khem Raj, Alexander Kanavin Cc: Patches and discussions about the oe-core layer On 2019年02月02日 06:32, Alejandro Enedino Hernandez Samaniego wrote: > On 2/1/19 12:49 PM, Khem Raj wrote: >> On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin<alex.kanavin@gmail.com> wrote: >>> I took the same approach as the recent perl upgrade: write recipe from scratch, >>> taking the pieces from the old recipe only when they were proven to be necessary. >>> >>> The pgo, manifest and ptest features are all preserved. >>> >>> New features: >>> >>> - native and target recipes are now unified into one recipe >>> >>> - check_build_completeness.py runs right after do_compile() and verifies that >>> all optional modules have been built (a notorious source of regressions) >>> >>> - a new approach to sysconfig.py and distutils/sysconfig.py returning values >>> appropriate for native or target builds: we copy the configuration file to a >>> separate folder, add that folder to sys.path (through environment variable >>> that differs between native and target builds), and point python to the file >>> through another environment variable. >>> >> Thanks, once current cycle is over for OE, I would like to give this a >> whirl and see how it goes >> >>> Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> --- >>> meta/classes/python3-dir.bbclass | 2 +- >>> meta/classes/python3native.bbclass | 2 + >>> ...ib-termcap-to-linker-flags-to-avoid-.patch | 25 + >>> ...hell-version-of-python-config-that-w.patch | 35 + >>> ...-qemu-wrapper-when-gathering-profile.patch | 25 + >>> ...fig-append-STAGING_LIBDIR-python-sys.patch | 42 + >>> ...tutils-prefix-is-inside-staging-area.patch | 54 + >>> .../python3/avoid_warning_about_tkinter.patch | 36 + >>> .../python-sanity/python3/cgi_py.patch | 32 + >>> .../python3/check_build_completeness.py | 17 + >>> .../python-sanity/python3/create_manifest3.py | 433 ++++++ >>> .../python-sanity/python3/get_module_deps3.py | 146 ++ >>> .../python-sanity/python3/python-config.patch | 46 + >>> .../python3/python3-manifest.json | 1227 +++++++++++++++++ >>> .../python-sanity/python3/run-ptest | 3 + >>> .../python-sanity/python3_3.7.2.bb | 281 ++++ >>> 16 files changed, 2405 insertions(+), 1 deletion(-) >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest >>> create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb > > Thanks Alex!, I looked at this briefly and it looks good. > > Do we want to keep the recipe at recipes-devtools-python-sanity?, also I think the PACKAGECONFIG for tk is broken but thats probably tk's fault, Hi Alejandro, What do you mean "the PACKAGECONFIG for tk is broken"? Thanks, > and, since tk is part of meta-oe only, I am wondering if it would make sense to have a bbappend there with the PACKAGECONFIG there, > since we're depending on something that were actually not holding? > > Alejandro > > >>> >>> diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass >>> index 06bb046d9c2..7dd130bad99 100644 >>> --- a/meta/classes/python3-dir.bbclass >>> +++ b/meta/classes/python3-dir.bbclass >>> @@ -1,4 +1,4 @@ >>> -PYTHON_BASEVERSION = "3.5" >>> +PYTHON_BASEVERSION = "3.7" >>> PYTHON_ABI = "m" >>> PYTHON_DIR = "python${PYTHON_BASEVERSION}" >>> PYTHON_PN = "python3" >>> diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass >>> index da12a714703..a3acaf61bbd 100644 >>> --- a/meta/classes/python3native.bbclass >>> +++ b/meta/classes/python3native.bbclass >>> @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native " >>> export STAGING_INCDIR >>> export STAGING_LIBDIR >>> >>> +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" >>> + >>> # suppress host user's site-packages dirs. >>> export PYTHONNOUSERSITE = "1" >>> >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> new file mode 100644 >>> index 00000000000..09f279ba1d7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch >>> @@ -0,0 +1,25 @@ >>> +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Fri, 25 Jan 2019 19:04:13 +0100 >>> +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host >>> + contamination >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> + >>> +--- >>> + setup.py | 1 - >>> + 1 file changed, 1 deletion(-) >>> + >>> +diff --git a/setup.py b/setup.py >>> +index b4357e3..fbec00d 100644 >>> +--- a/setup.py >>> ++++ b/setup.py >>> +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext): >>> + 'termcap'): >>> + readline_libs.append('termcap') >>> + exts.append( Extension('readline', ['readline.c'], >>> +- library_dirs=['/usr/lib/termcap'], >>> + extra_link_args=readline_extra_link_args, >>> + libraries=readline_libs) ) >>> + else: >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> new file mode 100644 >>> index 00000000000..83fd52d87f4 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch >>> @@ -0,0 +1,35 @@ >>> +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Tue, 29 Jan 2019 15:03:01 +0100 >>> +Subject: [PATCH] Do not use the shell version of python-config that was >>> + introduced in 3.4 >>> + >>> +Revert instead to the original python version: it has our tweaks and >>> +outputs directories correctly. >>> + >>> +Upstream-Status: Inappropriate [oe-specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> +--- >>> + Makefile.pre.in | 9 +++------ >>> + 1 file changed, 3 insertions(+), 6 deletions(-) >>> + >>> +diff --git a/Makefile.pre.in b/Makefile.pre.in >>> +index 2d2e11f..cc19942 100644 >>> +--- a/Makefile.pre.in >>> ++++ b/Makefile.pre.in >>> +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh >>> + sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py >>> + @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} >>> + LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config >>> +- @ # On Darwin, always use the python version of the script, the shell >>> +- @ # version doesn't use the compiler customizations that are provided >>> +- @ # in python (_osx_support.py). >>> +- @if test `uname -s` = Darwin; then \ >>> +- cp python-config.py python-config; \ >>> +- fi >>> ++ @ # In OpenEmbedded, always use the python version of the script, the shell >>> ++ @ # version is broken in multiple ways, and doesn't return correct directories >>> ++ cp python-config.py python-config >>> + >>> + >>> + # Install the include files >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> new file mode 100644 >>> index 00000000000..fa7735ff93e >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch >>> @@ -0,0 +1,25 @@ >>> +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Wed, 30 Jan 2019 12:41:04 +0100 >>> +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> +--- >>> + Makefile.pre.in | 3 +-- >>> + 1 file changed, 1 insertion(+), 2 deletions(-) >>> + >>> +diff --git a/Makefile.pre.in b/Makefile.pre.in >>> +index a3a02a7..d5503dd 100644 >>> +--- a/Makefile.pre.in >>> ++++ b/Makefile.pre.in >>> +@@ -507,8 +507,7 @@ build_all_generate_profile: >>> + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" >>> + >>> + run_profile_task: >>> +- @ # FIXME: can't run for a cross build >>> +- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true >>> ++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true >>> + >>> + build_all_merge_profile: >>> + $(LLVM_PROF_MERGER) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> new file mode 100644 >>> index 00000000000..2c5b76f131c >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch >>> @@ -0,0 +1,42 @@ >>> +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001 >>> +From: Alexander Kanavin<alex.kanavin@gmail.com> >>> +Date: Thu, 31 Jan 2019 16:46:30 +0100 >>> +Subject: [PATCH] distutils/sysconfig: append >>> + STAGING_LIBDIR/python-sysconfigdata to sys.path >>> + >>> +So that target configuration can be used when running native python >>> + >>> +Upstream-Status: Inappropriate [oe-core specific] >>> +Signed-off-by: Alexander Kanavin<alex.kanavin@gmail.com> >>> + >>> +--- >>> + Lib/distutils/sysconfig.py | 2 ++ >>> + Lib/sysconfig.py | 2 ++ >>> + 2 files changed, 4 insertions(+) >>> + >>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >>> +index e07a6c8..6b8c129 100644 >>> +--- a/Lib/distutils/sysconfig.py >>> ++++ b/Lib/distutils/sysconfig.py >>> +@@ -421,6 +421,8 @@ def _init_posix(): >>> + platform=sys.platform, >>> + multiarch=getattr(sys.implementation, '_multiarch', ''), >>> + )) >>> ++ if 'STAGING_LIBDIR' in os.environ: >>> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >>> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >>> + build_time_vars = _temp.build_time_vars >>> + global _config_vars >>> +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py >>> +index 9ee4d31..e586abd 100644 >>> +--- a/Lib/sysconfig.py >>> ++++ b/Lib/sysconfig.py >>> +@@ -412,6 +412,8 @@ def _init_posix(vars): >>> + """Initialize the module as appropriate for POSIX systems.""" >>> + # _sysconfigdata is generated at build time, see _generate_posix_vars() >>> + name = _get_sysconfigdata_name() >>> ++ if 'STAGING_LIBDIR' in os.environ: >>> ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >>> + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) >>> + build_time_vars = _temp.build_time_vars >>> + vars.update(build_time_vars) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> new file mode 100644 >>> index 00000000000..abf08f59dd7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch >>> @@ -0,0 +1,54 @@ >>> +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001 >>> +From: Khem Raj<raj.khem@gmail.com> >>> +Date: Tue, 14 May 2013 15:00:26 -0700 >>> +Subject: [PATCH] python3: Add target and native recipes >>> + >>> +Upstream-Status: Inappropriate [embedded specific] >>> + >>> +02/2015 Rebased for Python 3.4.2 >>> + >>> +# The proper prefix is inside our staging area. >>> +# Signed-Off: Michael 'Mickey' Lauer<mickey@vanille-media.de> >>> +# Signed-off-by: Phil Blundell<philb@gnu.org> >>> +# Signed-off-by: Khem Raj<raj.khem@gmail.com> >>> +# Signed-off-by: Alejandro Hernandez<alejandro.hernandez@linux.intel.com> >>> + >>> +--- >>> + Lib/distutils/sysconfig.py | 10 ++++++++-- >>> + 1 file changed, 8 insertions(+), 2 deletions(-) >>> + >>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py >>> +index 6b8c129..3ca7f79 100644 >>> +--- a/Lib/distutils/sysconfig.py >>> ++++ b/Lib/distutils/sysconfig.py >>> +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): >>> + If 'prefix' is supplied, use it instead of sys.base_prefix or >>> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >>> + """ >>> +- if prefix is None: >>> ++ if prefix is None and os.environ['STAGING_INCDIR'] != "": >>> ++ prefix = os.environ['STAGING_INCDIR'].rstrip('include') >>> ++ elif prefix is None: >>> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >>> + if os.name == "posix": >>> + if python_build: >>> +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >>> + If 'prefix' is supplied, use it instead of sys.base_prefix or >>> + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. >>> + """ >>> ++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] >>> ++ if prefix is None and os.environ['STAGING_LIBDIR'] != "": >>> ++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) >>> ++ >>> + if prefix is None: >>> + if standard_lib: >>> + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX >>> +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): >>> + >>> + if os.name == "posix": >>> + libpython = os.path.join(prefix, >>> +- "lib", "python" + get_python_version()) >>> ++ lib_basename, "python" + get_python_version()) >>> + if standard_lib: >>> + return libpython >>> + else: >>> diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> new file mode 100644 >>> index 00000000000..24e67b4ca14 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch >>> @@ -0,0 +1,36 @@ >>> +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001 >>> +From: Andrei Gherzan<andrei@gherzan.ro> >>> +Date: Mon, 28 Jan 2019 15:57:54 +0000 >>> +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet >>> + integrated in yocto so we skip the check for this module. Avoid a warning by >>> + not adding this module to missing variable. >>> + >>> +Upstream-Status: Inappropriate [distribution] >>> + >>> +Also simply disable the tk module since its not in DEPENDS. >>> +Signed-off-by: Andrei Gherzan<andrei@gherzan.ro> >>> + >>> +--- >>> + setup.py | 8 +++++--- >>> + 1 file changed, 5 insertions(+), 3 deletions(-) >>> + >>> +diff --git a/setup.py b/setup.py >>> +index fbec00d..b7a36a6 100644 >>> +--- a/setup.py >>> ++++ b/setup.py >>> +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext): >>> + self.extensions.extend(exts) >>> + >>> + # Call the method for detecting whether _tkinter can be compiled >>> +- self.detect_tkinter(inc_dirs, lib_dirs) >>> ++ # self.detect_tkinter(inc_dirs, lib_dirs) >>> + >>> +- if '_tkinter' not in [e.name for e in self.extensions]: >>> +- missing.append('_tkinter') >>> ++ # tkinter module will not be avalaible as yocto >>> ++ # doesn't have tk integrated (yet) >>> ++ #if '_tkinter' not in [e.name for e in self.extensions]: >>> ++ # missing.append('_tkinter') >>> + >>> + # Build the _uuid module if possible >>> + uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"]) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> new file mode 100644 >>> index 00000000000..6c4ba54320b >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch >>> @@ -0,0 +1,32 @@ >>> +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001 >>> +From: Mark Hatle<mark.hatle@windriver.com> >>> +Date: Wed, 21 Sep 2011 20:55:33 -0500 >>> +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment >>> + >>> +Upstream-Status: Inappropriate [distribution] >>> + >>> +Signed-off-by: Mark Hatle<mark.hatle@windriver.com> >>> + >>> +--- >>> + Lib/cgi.py | 11 +---------- >>> + 1 file changed, 1 insertion(+), 10 deletions(-) >>> + >>> +diff --git a/Lib/cgi.py b/Lib/cgi.py >>> +index 8cf6687..094c7b4 100755 >>> +--- a/Lib/cgi.py >>> ++++ b/Lib/cgi.py >>> +@@ -1,13 +1,4 @@ >>> +-#! /usr/local/bin/python >>> +- >>> +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is >>> +-# intentionally NOT "/usr/bin/env python". On many systems >>> +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI >>> +-# scripts, and /usr/local/bin is the default directory where Python is >>> +-# installed, so /usr/bin/env would be unable to find python. Granted, >>> +-# binary installations by Linux vendors often install Python in >>> +-# /usr/bin. So let those vendors patch cgi.py to match their choice >>> +-# of installation. >>> ++#! /usr/bin/env python >>> + >>> + """Support module for CGI (Common Gateway Interface) scripts. >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> new file mode 100755 >>> index 00000000000..a1eace3f571 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py >>> @@ -0,0 +1,17 @@ >>> +#!/usr/bin/env python3 >>> +import sys >>> +logfile = open(sys.argv[1]).read() >>> + >>> +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found") >>> +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.") >>> +if necessary_bits != -1: >>> + print("%s" %(logfile[necessary_bits:to_find_bits])) >>> + >>> +failed_to_build = logfile.find("Failed to build these modules:") >>> +if failed_to_build != -1: >>> + failed_to_build_end = logfile.find("\n\n", failed_to_build) >>> + print("%s" %(logfile[failed_to_build:failed_to_build_end])) >>> + >>> +if necessary_bits != -1 or failed_to_build != -1: >>> + sys.exit(1) >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> new file mode 100644 >>> index 00000000000..4da02a2991a >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py >>> @@ -0,0 +1,433 @@ >>> +# This script is used as a bitbake task to create a new python manifest >>> +# $ bitbake python -c create_manifest >>> +# >>> +# Our goal is to keep python-core as small as posible and add other python >>> +# packages only when the user needs them, hence why we split upstream python >>> +# into several packages. >>> +# >>> +# In a very simplistic way what this does is: >>> +# Launch python and see specifically what is required for it to run at a minimum >>> +# >>> +# Go through the python-manifest file and launch a separate task for every single >>> +# one of the files on each package, this task will check what was required for that >>> +# specific module to run, these modules will be called dependencies. >>> +# The output of such task will be a list of the modules or dependencies that were >>> +# found for that file. >>> +# >>> +# Such output will be parsed by this script, we will look for each dependency on the >>> +# manifest and if we find that another package already includes it, then we will add >>> +# that package as an RDEPENDS to the package we are currently checking; in case we dont >>> +# find the current dependency on any other package we will add it to the current package >>> +# as part of FILES. >>> +# >>> +# >>> +# This way we will create a new manifest from the data structure that was built during >>> +# this process, on this new manifest each package will contain specifically only >>> +# what it needs to run. >>> +# >>> +# There are some caveats which we try to deal with, such as repeated files on different >>> +# packages, packages that include folders, wildcards, and special packages. >>> +# Its also important to note that this method only works for python files, and shared >>> +# libraries. Static libraries, header files and binaries need to be dealt with manually. >>> +# >>> +# This script differs from its python2 version mostly on how shared libraries are handled >>> +# The manifest file for python3 has an extra field which contains the cached files for >>> +# each package. >>> +# Tha method to handle cached files does not work when a module includes a folder which >>> +# itself contains the pycache folder, gladly this is almost never the case. >>> +# >>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> >>> + >>> + >>> +import sys >>> +import subprocess >>> +import json >>> +import os >>> +import collections >>> + >>> +# Get python version from ${PYTHON_MAJMIN} >>> +pyversion = str(sys.argv[1]) >>> + >>> +# Hack to get native python search path (for folders), not fond of it but it works for now >>> +pivot = 'recipe-sysroot-native' >>> +for p in sys.path: >>> + if pivot in p: >>> + nativelibfolder = p[:p.find(pivot)+len(pivot)] >>> + >>> +# Empty dict to hold the whole manifest >>> +new_manifest = collections.OrderedDict() >>> + >>> +# Check for repeated files, folders and wildcards >>> +allfiles = [] >>> +repeated = [] >>> +wildcards = [] >>> + >>> +hasfolders = [] >>> +allfolders = [] >>> + >>> +def isFolder(value): >>> + value = value.replace('${PYTHON_MAJMIN}',pyversion) >>> + if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): >>> + return True >>> + else: >>> + return False >>> + >>> +def isCached(item): >>> + if '__pycache__' in item: >>> + return True >>> + else: >>> + return False >>> + >>> +def prepend_comments(comments, json_manifest): >>> + with open(json_manifest, 'r+') as manifest: >>> + json_contents = manifest.read() >>> + manifest.seek(0, 0) >>> + manifest.write(comments + json_contents) >>> + >>> +# Read existing JSON manifest >>> +with open('python3-manifest.json') as manifest: >>> + # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker >>> + manifest_str = manifest.read() >>> + json_start = manifest_str.find('# EOC') + 6 # EOC + \n >>> + manifest.seek(0) >>> + comments = manifest.read(json_start) >>> + manifest_str = manifest.read() >>> + old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >>> + >>> +# >>> +# First pass to get core-package functionality, because we base everything on the fact that core is actually working >>> +# Not exactly the same so it should not be a function >>> +# >>> + >>> +print ('Getting dependencies for package: core') >>> + >>> + >>> +# This special call gets the core dependencies and >>> +# appends to the old manifest so it doesnt hurt what it >>> +# currently holds. >>> +# This way when other packages check for dependencies >>> +# on the new core package, they will still find them >>> +# even when checking the old_manifest >>> + >>> +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') >>> +for coredep in output.split(): >>> + coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + if isCached(coredep): >>> + if coredep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(coredep) >>> + else: >>> + if coredep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(coredep) >>> + >>> + >>> +# The second step is to loop through the existing files contained in the core package >>> +# according to the old manifest, identify if they are modules, or some other type >>> +# of file that we cant import (directories, binaries, configs) in which case we >>> +# can only assume they were added correctly (manually) so we ignore those and >>> +# pass them to the manifest directly. >>> + >>> +for filedep in old_manifest['core']['files']: >>> + if isFolder(filedep): >>> + if isCached(filedep): >>> + if filedep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(filedep) >>> + else: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + if '${bindir}' in filedep: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + if filedep == '': >>> + continue >>> + if '${includedir}' in filedep: >>> + if filedep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(filedep) >>> + continue >>> + >>> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >>> + pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] >>> + >>> + >>> + # We now know that were dealing with a python module, so we can import it >>> + # and check what its dependencies are. >>> + # We launch a separate task for each module for deterministic behavior. >>> + # Each module will only import what is necessary for it to work in specific. >>> + # The output of each task will contain each module's dependencies >>> + >>> + print ('Getting dependencies for module: %s' % pymodule) >>> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >>> + print ('The following dependencies were found for module %s:\n' % pymodule) >>> + print (output) >>> + >>> + >>> + for pymodule_dep in output.split(): >>> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + >>> + if isCached(pymodule_dep): >>> + if pymodule_dep not in old_manifest['core']['cached']: >>> + old_manifest['core']['cached'].append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in old_manifest['core']['files']: >>> + old_manifest['core']['files'].append(pymodule_dep) >>> + >>> + >>> +# At this point we are done with the core package. >>> +# The old_manifest dictionary is updated only for the core package because >>> +# all others will use this a base. >>> + >>> + >>> +# To improve the script speed, we check which packages contain directories >>> +# since we will be looping through (only) those later. >>> +for pypkg in old_manifest: >>> + for filedep in old_manifest[pypkg]['files']: >>> + if isFolder(filedep): >>> + print ('%s is a folder' % filedep) >>> + if pypkg not in hasfolders: >>> + hasfolders.append(pypkg) >>> + if filedep not in allfolders: >>> + allfolders.append(filedep) >>> + >>> + >>> + >>> +# This is the main loop that will handle each package. >>> +# It works in a similar fashion than the step before, but >>> +# we will now be updating a new dictionary that will eventually >>> +# become the new manifest. >>> +# >>> +# The following loops though all packages in the manifest, >>> +# through all files on each of them, and checks whether or not >>> +# they are modules and can be imported. >>> +# If they can be imported, then it checks for dependencies for >>> +# each of them by launching a separate task. >>> +# The output of that task is then parsed and the manifest is updated >>> +# accordingly, wether it should add the module on FILES for the current package >>> +# or if that module already belongs to another package then the current one >>> +# will RDEPEND on it >>> + >>> +for pypkg in old_manifest: >>> + # Use an empty dict as data structure to hold data for each package and fill it up >>> + new_manifest[pypkg] = collections.OrderedDict() >>> + new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] >>> + new_manifest[pypkg]['rdepends'] = [] >>> + new_manifest[pypkg]['files'] = [] >>> + new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] >>> + >>> + # All packages should depend on core >>> + if pypkg != 'core': >>> + new_manifest[pypkg]['rdepends'].append('core') >>> + new_manifest[pypkg]['cached'] = [] >>> + >>> + print('\n') >>> + print('--------------------------') >>> + print ('Handling package %s' % pypkg) >>> + print('--------------------------') >>> + >>> + # Handle special cases, we assume that when they were manually added >>> + # to the manifest we knew what we were doing. >>> + special_packages = ['misc', 'modules', 'dev', 'tests'] >>> + if pypkg in special_packages or 'staticdev' in pypkg: >>> + print('Passing %s package directly' % pypkg) >>> + new_manifest[pypkg] = old_manifest[pypkg] >>> + continue >>> + >>> + for filedep in old_manifest[pypkg]['files']: >>> + # We already handled core on the first pass, we can ignore it now >>> + if pypkg == 'core': >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + >>> + # Handle/ignore what we cant import >>> + if isFolder(filedep): >>> + new_manifest[pypkg]['files'].append(filedep) >>> + # Asyncio (and others) are both the package and the folder name, we should not skip those... >>> + path,mod = os.path.split(filedep) >>> + if mod != pypkg: >>> + continue >>> + if '${bindir}' in filedep: >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + if filedep == '': >>> + continue >>> + if '${includedir}' in filedep: >>> + if filedep not in new_manifest[pypkg]['files']: >>> + new_manifest[pypkg]['files'].append(filedep) >>> + continue >>> + >>> + # Get actual module name , shouldnt be affected by libdir/bindir, etc. >>> + # We need to check if the imported module comes from another (e.g. sqlite3.dump) >>> + path,pymodule = os.path.split(filedep) >>> + path = os.path.basename(path) >>> + pymodule = os.path.splitext(os.path.basename(pymodule))[0] >>> + >>> + # If this condition is met, it means we need to import it from another module >>> + # or its the folder itself (e.g. unittest) >>> + if path == pypkg: >>> + if pymodule: >>> + pymodule = path + '.' + pymodule >>> + else: >>> + pymodule = path >>> + >>> + >>> + >>> + # We now know that were dealing with a python module, so we can import it >>> + # and check what its dependencies are. >>> + # We launch a separate task for each module for deterministic behavior. >>> + # Each module will only import what is necessary for it to work in specific. >>> + # The output of each task will contain each module's dependencies >>> + >>> + print ('\nGetting dependencies for module: %s' % pymodule) >>> + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') >>> + print ('The following dependencies were found for module %s:\n' % pymodule) >>> + print (output) >>> + >>> + reportFILES = [] >>> + reportRDEPS = [] >>> + >>> + for pymodule_dep in output.split(): >>> + >>> + # Warning: This first part is ugly >>> + # One of the dependencies that was found, could be inside of one of the folders included by another package >>> + # We need to check if this happens so we can add the package containing the folder as an rdependency >>> + # e.g. Folder encodings contained in codecs >>> + # This would be solved if no packages included any folders >>> + >>> + # This can be done in two ways: >>> + # 1 - We assume that if we take out the filename from the path we would get >>> + # the folder string, then we would check if folder string is in the list of folders >>> + # This would not work if a package contains a folder which contains another folder >>> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >>> + # folder_string would not match any value contained in the list of folders >>> + # >>> + # 2 - We do it the other way around, checking if the folder is contained in the path >>> + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 >>> + # is folder_string inside path/folder1/folder2/filename?, >>> + # Yes, it works, but we waste a couple of milliseconds. >>> + >>> + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') >>> + inFolders = False >>> + for folder in allfolders: >>> + # The module could have a directory named after it, e.g. xml, if we take out the filename from the path >>> + # we'll end up with ${libdir}, and we want ${libdir}/xml >>> + if isFolder(pymodule_dep): >>> + check_path = pymodule_dep >>> + else: >>> + check_path = os.path.dirname(pymodule_dep) >>> + if folder in check_path : >>> + inFolders = True # Did we find a folder? >>> + folderFound = False # Second flag to break inner for >>> + # Loop only through packages which contain folders >>> + for pypkg_with_folder in hasfolders: >>> + if (folderFound == False): >>> + # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) >>> + for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: >>> + if folder_dep == folder: >>> + print ('%s folder found in %s' % (folder, pypkg_with_folder)) >>> + folderFound = True >>> + if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: >>> + new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) >>> + else: >>> + break >>> + >>> + # A folder was found so we're done with this item, we can go on >>> + if inFolders: >>> + continue >>> + >>> + >>> + >>> + # No directories beyond this point >>> + # We might already have this module on the dictionary since it could depend on a (previously checked) module >>> + if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: >>> + # Handle core as a special package, we already did it so we pass it to NEW data structure directly >>> + if pypkg == 'core': >>> + print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) >>> + if pymodule_dep.endswith('*'): >>> + wildcards.append(pymodule_dep) >>> + if isCached(pymodule_dep): >>> + new_manifest[pypkg]['cached'].append(pymodule_dep) >>> + else: >>> + new_manifest[pypkg]['files'].append(pymodule_dep) >>> + >>> + # Check for repeated files >>> + if pymodule_dep not in allfiles: >>> + allfiles.append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in repeated: >>> + repeated.append(pymodule_dep) >>> + else: >>> + >>> + >>> + # Last step: Figure out if we this belongs to FILES or RDEPENDS >>> + # We check if this module is already contained on another package, so we add that one >>> + # as an RDEPENDS, or if its not, it means it should be contained on the current >>> + # package, and we should add it to FILES >>> + for possible_rdep in old_manifest: >>> + # Debug >>> + # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) >>> + if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: >>> + # Since were nesting, we need to check its not the same pypkg >>> + if(possible_rdep != pypkg): >>> + if possible_rdep not in new_manifest[pypkg]['rdepends']: >>> + # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs >>> + reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) >>> + new_manifest[pypkg]['rdepends'].append(possible_rdep) >>> + break >>> + else: >>> + >>> + # Since this module wasnt found on another package, it is not an RDEP, >>> + # so we add it to FILES for this package. >>> + # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) >>> + if os.path.basename(pymodule_dep) != pypkg: >>> + reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) >>> + if isCached(pymodule_dep): >>> + new_manifest[pypkg]['cached'].append(pymodule_dep) >>> + else: >>> + new_manifest[pypkg]['files'].append(pymodule_dep) >>> + if pymodule_dep.endswith('*'): >>> + wildcards.append(pymodule_dep) >>> + if pymodule_dep not in allfiles: >>> + allfiles.append(pymodule_dep) >>> + else: >>> + if pymodule_dep not in repeated: >>> + repeated.append(pymodule_dep) >>> + >>> + print('\n') >>> + print('#################################') >>> + print('Summary for module %s' % pymodule) >>> + print('FILES found for module %s:' % pymodule) >>> + print(''.join(reportFILES)) >>> + print('RDEPENDS found for module %s:' % pymodule) >>> + print(''.join(reportRDEPS)) >>> + print('#################################') >>> + >>> +print('The following FILES contain wildcards, please check if they are necessary') >>> +print(wildcards) >>> +print('The following FILES contain folders, please check if they are necessary') >>> +print(hasfolders) >>> + >>> + >>> +# Sort it just so it looks nicer >>> +for pypkg in new_manifest: >>> + new_manifest[pypkg]['files'].sort() >>> + new_manifest[pypkg]['cached'].sort() >>> + new_manifest[pypkg]['rdepends'].sort() >>> + >>> +# Create the manifest from the data structure that was built >>> +with open('python3-manifest.json.new','w') as outfile: >>> + json.dump(new_manifest,outfile, indent=4) >>> + outfile.write('\n') >>> + >>> +prepend_comments(comments,'python3-manifest.json.new') >>> + >>> +if (repeated): >>> + error_msg = '\n\nERROR:\n' >>> + error_msg += 'The following files are repeated (contained in more than one package),\n' >>> + error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' >>> + error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' >>> + error_msg += '\n'.join(repeated) >>> + error_msg += '\n' >>> + sys.exit(error_msg) >>> + >>> diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> new file mode 100644 >>> index 00000000000..fd12baad84e >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py >>> @@ -0,0 +1,146 @@ >>> +# This script is launched on separate task for each python module >>> +# It checks for dependencies for that specific module and prints >>> +# them out, the output of this execution will have all dependencies >>> +# for a specific module, which will be parsed an dealt on create_manifest.py >>> +# >>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29"<aehs29@gmail.com> >>> + >>> +# We can get a log per module, for all the dependencies that were found, but its messy. >>> +debug=False >>> + >>> +import sys >>> + >>> +# We can get a list of the modules which are currently required to run python >>> +# so we run python-core and get its modules, we then import what we need >>> +# and check what modules are currently running, if we substract them from the >>> +# modules we had initially, we get the dependencies for the module we imported. >>> + >>> +# We use importlib to achieve this, so we also need to know what modules importlib needs >>> +import importlib >>> + >>> +core_deps=set(sys.modules) >>> + >>> +def fix_path(dep_path): >>> + import os >>> + # We DONT want the path on our HOST system >>> + pivot='recipe-sysroot-native' >>> + dep_path=dep_path[dep_path.find(pivot)+len(pivot):] >>> + >>> + if '/usr/bin' in dep_path: >>> + dep_path = dep_path.replace('/usr/bin''${bindir}') >>> + >>> + # Handle multilib, is there a better way? >>> + if '/usr/lib32' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib32','${libdir}') >>> + if '/usr/lib64' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib64','${libdir}') >>> + if '/usr/lib' in dep_path: >>> + dep_path = dep_path.replace('/usr/lib','${libdir}') >>> + if '/usr/include' in dep_path: >>> + dep_path = dep_path.replace('/usr/include','${includedir}') >>> + if '__init__.' in dep_path: >>> + dep_path = os.path.split(dep_path)[0] >>> + return dep_path >>> + >>> + >>> +# Module to import was passed as an argument >>> +current_module = str(sys.argv[1]).rstrip() >>> +if(debug==True): >>> + log = open('log_%s' % current_module,'w') >>> + log.write('Module %s generated the following dependencies:\n' % current_module) >>> +try: >>> + importlib.import_module('%s' % current_module) >>> +except ImportError as e: >>> + if (debug==True): >>> + log.write('Module was not found') >>> + pass >>> + >>> + >>> +# Get current module dependencies, dif will contain a list of specific deps for this module >>> +module_deps=set(sys.modules) >>> + >>> +# We handle the core package (1st pass on create_manifest.py) as a special case >>> +if current_module == 'python-core-package': >>> + dif = core_deps >>> +else: >>> + # We know this is not the core package, so there must be a difference. >>> + dif = module_deps-core_deps >>> + >>> + >>> +# Check where each dependency came from >>> +for item in dif: >>> + dep_path='' >>> + try: >>> + if (debug==True): >>> + log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') >>> + dep_path = sys.modules['%s' % item].__file__ >>> + except AttributeError as e: >>> + # Deals with thread (builtin module) not having __file__ attribute >>> + if debug==True: >>> + log.write(item + ' ') >>> + log.write(str(e)) >>> + log.write('\n') >>> + pass >>> + except NameError as e: >>> + # Deals with NameError: name 'dep_path' is not defined >>> + # because module is not found (wasn't compiled?), e.g. bddsm >>> + if (debug==True): >>> + log.write(item+' ') >>> + log.write(str(e)) >>> + pass >>> + >>> + # Site-customize is a special case since we (OpenEmbedded) put it there manually >>> + if 'sitecustomize' in dep_path: >>> + dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' >>> + # Prints out result, which is what will be used by create_manifest >>> + print (dep_path) >>> + continue >>> + >>> + dep_path = fix_path(dep_path) >>> + >>> + import sysconfig >>> + soabi=sysconfig.get_config_var('SOABI') >>> + # Check if its a shared library and deconstruct it >>> + if soabi in dep_path: >>> + if (debug==True): >>> + log.write('Shared library found in %s' % dep_path) >>> + dep_path = dep_path.replace(soabi,'*') >>> + print (dep_path) >>> + continue >>> + >>> + if (debug==True): >>> + log.write(dep_path+'\n') >>> + # Prints out result, which is what will be used by create_manifest >>> + print (dep_path) >>> + >>> + >>> + import imp >>> + cpython_tag = imp.get_tag() >>> + cached='' >>> + # Theres no naive way to find *.pyc files on python3 >>> + try: >>> + if (debug==True): >>> + log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') >>> + cached = sys.modules['%s' % item].__cached__ >>> + except AttributeError as e: >>> + # Deals with thread (builtin module) not having __cached__ attribute >>> + if debug==True: >>> + log.write(item + ' ') >>> + log.write(str(e)) >>> + log.write('\n') >>> + pass >>> + except NameError as e: >>> + # Deals with NameError: name 'cached' is not defined >>> + if (debug==True): >>> + log.write(item+' ') >>> + log.write(str(e)) >>> + pass >>> + if cached is not None: >>> + if (debug==True): >>> + log.write(cached) >>> + cached = fix_path(cached) >>> + cached = cached.replace(cpython_tag,'*') >>> + print (cached) >>> + >>> +if debug==True: >>> + log.close() >>> diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch >>> new file mode 100644 >>> index 00000000000..f23b8b7df06 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch >>> @@ -0,0 +1,46 @@ >>> +python-config: Revert to using distutils.sysconfig >>> + >>> +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in >>> + >>> +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig >>> + >>> +affect the native runtime as well as cross building. Use the old, patched >>> +implementation which returns paths in the staging directory and for the target, >>> +as appropriate. >>> + >>> +Upstream-Status: Inappropriate [Embedded Specific] >>> + >>> +Signed-off-by: Tyler Hall<tylerwhall@gmail.com> >>> +: >>> +Index: Python-3.3.3/Misc/python-config.in >>> +=================================================================== >>> +--- Python-3.3.3.orig/Misc/python-config.in >>> ++++ Python-3.3.3/Misc/python-config.in >>> +@@ -4,7 +4,7 @@ >>> + import getopt >>> + import os >>> + import sys >>> +-import sysconfig >>> ++from distutils import sysconfig >>> + >>> + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', >>> + 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] >>> +@@ -32,14 +32,14 @@ if '--help' in opt_flags: >>> + >>> + for opt in opt_flags: >>> + if opt == '--prefix': >>> +- print(sysconfig.get_config_var('prefix')) >>> ++ print(sysconfig.PREFIX) >>> + >>> + elif opt == '--exec-prefix': >>> +- print(sysconfig.get_config_var('exec_prefix')) >>> ++ print(sysconfig.EXEC_PREFIX) >>> + >>> + elif opt in ('--includes', '--cflags'): >>> +- flags = ['-I' + sysconfig.get_path('include'), >>> +- '-I' + sysconfig.get_path('platinclude')] >>> ++ flags = ['-I' + sysconfig.get_python_inc(), >>> ++ '-I' + sysconfig.get_python_inc(plat_specific=True)] >>> + if opt == '--cflags': >>> + flags.extend(getvar('CFLAGS').split()) >>> + print(' '.join(flags)) >>> diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> new file mode 100644 >>> index 00000000000..82c7075f0e7 >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json >>> @@ -0,0 +1,1227 @@ >>> +# DO NOT (entirely) modify this file manually, please read. >>> +# >>> +# IMPORTANT NOTE: >>> +# Please keep in mind that the create_manifest task relies on the fact the the >>> +# target and native Python packages are the same, and it also needs to be executed >>> +# with a fully working native package (with all the PACKAGECONFIGs enabled and all >>> +# and all the modules should be working, check log.do_compile), otherwise the script >>> +# will fail to find dependencies correctly, this note is valid either if you are >>> +# upgrading to a new Python version or adding a new package. >>> +# >>> +# >>> +# If you are adding a new package please follow the next steps: >>> +# How to add a new package: >>> +# - If a user wants to add a new package all that has to be done is: >>> +# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, >>> +# fill up the SUMMARY section as well, the script should handle all the rest. >>> +# >>> +# Real example: >>> +# We want to add a web browser package, including the file webbrowser.py >>> +# which at the moment is on python3-misc. >>> +# "webbrowser": { >>> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], >>> +# "rdepends": [], >>> +# "summary": "Python Web Browser support"} >>> +# >>> +# * Note that the rdepends field was left empty >>> +# >>> +# We run $ bitbake python3 -c create_manifest and the resulting manifest >>> +# should be completed after a few seconds, showing something like: >>> +# "webbrowser": { >>> +# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], >>> +# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], >>> +# "summary": "Python Web Browser support"} >>> +# >>> +# >>> +# If you are upgrading Python to a new version please follow the next steps: >>> +# After each Python upgrade, the create_manifest task should be executed, because we >>> +# don't control what changes on upstream Python, so, some module dependency >>> +# might have changed without us realizing it, a certain module can either have >>> +# more or less dependencies, or could be depending on a new file that was just >>> +# created on the new release and for obvious reasons we wouldn't have it on our >>> +# old manifest, all of these issues would cause runtime errors on our system. >>> +# >>> +# - Upgrade both the native and target Python packages to a new version >>> +# - Run the create_manifest task for the target Python package as its shown below: >>> +# >>> +# $ bitbake python3 -c create_manifest >>> +# >>> +# This will automatically replace your manifest file located under the Python directory >>> +# with an new one, which contains the new dependencies (if any). >>> +# >>> +# Several things could have gone wrong here, I will try to explain a few: >>> +# >>> +# a) A new file was introduced on this release, e.g. sha3*.so: >>> +# The task will check what its needed to import every module, more than one module would >>> +# would probably depend on sha3*.so, although only one module should contain it. >>> +# >>> +# After running the task, the new manifest will have the sha3*.so file on more than one >>> +# module, you need to manually decide which one of them should get it and delete it from >>> +# the others, for example sha3*.so should likely be on ${PN}-crypt. >>> +# Once you have deleted from the others you need to run the create_manifest task again, >>> +# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be >>> +# good to go. >>> +# >>> +# b) The native package wasn't built correctly and its missing a certain module: >>> +# As mentioned before, you need to make sure the native package was built with all the modules >>> +# because it is used as base to build the manifest file, you need to manually check log.do_compile >>> +# since it won't error out the compile function if its only missing a couple of modules. >>> +# >>> +# e.g. missing the _uuid module, log.do_compile would show the following: >>> +# Python build finished successfully! >>> +# The necessary bits to build these optional modules were not found: >>> +# _uuid >>> +# >>> +# What will happen here is that the new manifest would not be aware that the _uuid module exists, so >>> +# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on >>> +# the misc package (which is where any file that doesn't belong anywhere else ends up). >>> +# >>> +# This will eventually cause runtime errors on our system if we don't include the misc package on >>> +# on our image, because the _uuid files will be missing. >>> +# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be >>> +# detected correctly along with its dependencies, and we will get a working manifest. >>> +# >>> +# This is the reason why it is important to make sure we have a fully working native build, >>> +# so we can avoid these errors. >>> +# >>> +# >>> +# >>> +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST >>> +# EOC >>> +{ >>> + "tests": { >>> + "summary": "Python test suite", >>> + "rdepends": [ >>> + "core", >>> + "modules" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/*/test", >>> + "${libdir}/python${PYTHON_MAJMIN}/*/tests", >>> + "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", >>> + "${libdir}/python${PYTHON_MAJMIN}/test" >>> + ], >>> + "cached": [] >>> + }, >>> + "2to3": { >>> + "summary": "Python automated Python 2 to 3 code translator", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${bindir}/2to3-*", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib2to3" >>> + ], >>> + "cached": [] >>> + }, >>> + "asyncio": { >>> + "summary": "Python Asynchronous I/", >>> + "rdepends": [ >>> + "core", >>> + "io", >>> + "logging", >>> + "netclient", >>> + "numbers", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/asyncio", >>> + "${libdir}/python${PYTHON_MAJMIN}/concurrent", >>> + "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "audio": { >>> + "summary": "Python Audio Handling", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/chunk.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sunau.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/wave.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" >>> + ] >>> + }, >>> + "codecs": { >>> + "summary": "Python codec", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" >>> + ] >>> + }, >>> + "compile": { >>> + "summary": "Python bytecode compilation support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/compileall.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" >>> + ] >>> + }, >>> + "compression": { >>> + "summary": "Python high-level compression support", >>> + "rdepends": [ >>> + "core", >>> + "shell", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_compression.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/bz2.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/gzip.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lzma.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" >>> + ] >>> + }, >>> + "core": { >>> + "summary": "Python interpreter and core modules", >>> + "rdepends": [], >>> + "files": [ >>> + "${bindir}/python*[!-config]", >>> + "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserList.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/UserString.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/__future__.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/argparse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ast.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/bisect.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/code.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/codecs.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/codeop.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/configparser.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/copy.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/csv.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/dis.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/enum.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/functools.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/getopt.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/gettext.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/heapq.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/imp.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/inspect.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/io.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/keyword.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/linecache.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/locale.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/new.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/opcode.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/operator.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/optparse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/os.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/platform.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/re.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/selectors.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/signal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/site.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/stat.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/struct.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/symbol.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/threading.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/token.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/traceback.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/types.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/warnings.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/weakref.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" >>> + ] >>> + }, >>> + "crypt": { >>> + "summary": "Python basic cryptographic and hashing support", >>> + "rdepends": [ >>> + "core", >>> + "math", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/crypt.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" >>> + ] >>> + }, >>> + "ctypes": { >>> + "summary": "Python C types support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/ctypes", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "curses": { >>> + "summary": "Python curses support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/curses", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "datetime": { >>> + "summary": "Python calendar and time support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/calendar.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/datetime.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" >>> + ] >>> + }, >>> + "db": { >>> + "summary": "Python file-based database support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/dbm", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "debugger": { >>> + "summary": "Python debugger", >>> + "rdepends": [ >>> + "core", >>> + "pprint", >>> + "shell", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/bdb.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pdb.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" >>> + ] >>> + }, >>> + "dev": { >>> + "cached": [], >>> + "files": [ >>> + "${base_libdir}/*.a", >>> + "${base_libdir}/*.o", >>> + "${bindir}/python*-config", >>> + "${datadir}/aclocal", >>> + "${datadir}/pkgconfig", >>> + "${includedir}", >>> + "${libdir}/*.a", >>> + "${libdir}/*.la", >>> + "${libdir}/*.o", >>> + "${libdir}/lib*${SOLIBSDEV}", >>> + "${libdir}/pkgconfig", >>> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", >>> + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" >>> + ], >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "summary": "Python development package" >>> + }, >>> + "difflib": { >>> + "summary": "Python helpers for computing deltas between objects", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/difflib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" >>> + ] >>> + }, >>> + "distutils-staticdev": { >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" >>> + ], >>> + "rdepends": [ >>> + "distutils" >>> + ], >>> + "summary": "Python distribution utilities (static libraries)" >>> + }, >>> + "distutils": { >>> + "summary": "Python Distribution Utilities", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/distutils" >>> + ], >>> + "cached": [] >>> + }, >>> + "doctest": { >>> + "summary": "Python framework for running examples in docstrings", >>> + "rdepends": [ >>> + "core", >>> + "debugger", >>> + "difflib", >>> + "logging", >>> + "pprint", >>> + "shell", >>> + "stringold", >>> + "unittest" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/doctest.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" >>> + ] >>> + }, >>> + "email": { >>> + "summary": "Python email support", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "io", >>> + "math", >>> + "netclient" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/email", >>> + "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" >>> + ] >>> + }, >>> + "fcntl": { >>> + "summary": "Python's fcntl interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "gdbm": { >>> + "summary": "Python GNU database support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "html": { >>> + "summary": "Python HTML processing support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/formatter.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/html" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" >>> + ] >>> + }, >>> + "idle": { >>> + "summary": "Python Integrated Development Environment", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${bindir}/idle*", >>> + "${libdir}/python${PYTHON_MAJMIN}/idlelib" >>> + ], >>> + "cached": [] >>> + }, >>> + "image": { >>> + "summary": "Python graphical image handling", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" >>> + ] >>> + }, >>> + "io": { >>> + "summary": "Python low-level I/O", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "crypt", >>> + "math", >>> + "netclient", >>> + "shell", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/pipes.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/socket.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ssl.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" >>> + ] >>> + }, >>> + "json": { >>> + "summary": "Python JSON support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/json", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "logging": { >>> + "summary": "Python logging support", >>> + "rdepends": [ >>> + "core", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/logging" >>> + ], >>> + "cached": [] >>> + }, >>> + "mailbox": { >>> + "summary": "Python mailbox format support", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "fcntl", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" >>> + ] >>> + }, >>> + "math": { >>> + "summary": "Python math support", >>> + "rdepends": [ >>> + "core", >>> + "crypt" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/random.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" >>> + ] >>> + }, >>> + "mime": { >>> + "summary": "Python MIME handling APIs", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/quopri.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/uu.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" >>> + ] >>> + }, >>> + "mmap": { >>> + "summary": "Python memory-mapped file support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "modules": { >>> + "cached": [], >>> + "files": [], >>> + "rdepends": [ >>> + "2to3", >>> + "asyncio", >>> + "audio", >>> + "codecs", >>> + "compile", >>> + "compression", >>> + "core", >>> + "crypt", >>> + "ctypes", >>> + "curses", >>> + "datetime", >>> + "db", >>> + "debugger", >>> + "difflib", >>> + "distutils", >>> + "doctest", >>> + "email", >>> + "fcntl", >>> + "html", >>> + "idle", >>> + "image", >>> + "io", >>> + "json", >>> + "logging", >>> + "mailbox", >>> + "math", >>> + "mime", >>> + "mmap", >>> + "multiprocessing", >>> + "netclient", >>> + "netserver", >>> + "numbers", >>> + "pickle", >>> + "pkgutil", >>> + "plistlib", >>> + "pprint", >>> + "profile", >>> + "pydoc", >>> + "resource", >>> + "runpy", >>> + "shell", >>> + "smtpd", >>> + "sqlite3", >>> + "stringold", >>> + "syslog", >>> + "terminal", >>> + "threading", >>> + "tkinter", >>> + "typing", >>> + "unittest", >>> + "unixadmin", >>> + "venv", >>> + "xml", >>> + "xmlrpc" >>> + ], >>> + "summary": "All Python modules" >>> + }, >>> + "multiprocessing": { >>> + "summary": "Python multiprocessing support", >>> + "rdepends": [ >>> + "core", >>> + "io", >>> + "pickle" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" >>> + ], >>> + "cached": [] >>> + }, >>> + "netclient": { >>> + "summary": "Python Internet Protocol clients", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "io", >>> + "math", >>> + "mime", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/base64.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/hmac.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/http", >>> + "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/poplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/urllib", >>> + "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", >>> + "${libdir}/python${PYTHON_MAJMIN}/uuid.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" >>> + ] >>> + }, >>> + "netserver": { >>> + "summary": "Python Internet Protocol servers", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "html", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "shell", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cgi.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" >>> + ] >>> + }, >>> + "numbers": { >>> + "summary": "Python number APIs", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/contextvars.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/decimal.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/fractions.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/numbers.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" >>> + ] >>> + }, >>> + "pickle": { >>> + "summary": "Python serialisation/persistence support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/pickle.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shelve.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" >>> + ] >>> + }, >>> + "pkgutil": { >>> + "summary": "Python package extension utility support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" >>> + ] >>> + }, >>> + "plistlib": { >>> + "summary": "Generate and parse Mac OS X .plist files", >>> + "rdepends": [ >>> + "core", >>> + "datetime", >>> + "xml" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" >>> + ] >>> + }, >>> + "pprint": { >>> + "summary": "Python pretty-print support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pprint.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" >>> + ] >>> + }, >>> + "profile": { >>> + "summary": "Python basic performance profiling support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/profile.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pstats.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" >>> + ] >>> + }, >>> + "pydoc": { >>> + "summary": "Python interactive help support", >>> + "rdepends": [ >>> + "core", >>> + "netclient", >>> + "pkgutil" >>> + ], >>> + "files": [ >>> + "${bindir}/pydoc*", >>> + "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" >>> + ] >>> + }, >>> + "resource": { >>> + "summary": "Python resource control interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "runpy": { >>> + "summary": "Python helper for locating/executing scripts in module namespace", >>> + "rdepends": [ >>> + "core", >>> + "pkgutil" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/runpy.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" >>> + ] >>> + }, >>> + "shell": { >>> + "summary": "Python shell-like functionality", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/cmd.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/glob.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shlex.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/shutil.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" >>> + ] >>> + }, >>> + "smtpd": { >>> + "summary": "Python Simple Mail Transport Daemon", >>> + "rdepends": [ >>> + "core", >>> + "crypt", >>> + "datetime", >>> + "email", >>> + "io", >>> + "math", >>> + "mime", >>> + "netclient", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${bindir}/smtpd.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" >>> + ] >>> + }, >>> + "sqlite3": { >>> + "summary": "Python Sqlite3 database support", >>> + "rdepends": [ >>> + "core", >>> + "datetime" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/sqlite3" >>> + ], >>> + "cached": [] >>> + }, >>> + "stringold": { >>> + "summary": "Python string APIs [deprecated]", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/string.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" >>> + ] >>> + }, >>> + "syslog": { >>> + "summary": "Python syslog interface", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" >>> + ], >>> + "cached": [] >>> + }, >>> + "terminal": { >>> + "summary": "Python terminal controlling support", >>> + "rdepends": [ >>> + "core", >>> + "io" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/pty.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/tty.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" >>> + ] >>> + }, >>> + "threading": { >>> + "summary": "Python threading & synchronization support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/queue.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" >>> + ] >>> + }, >>> + "tkinter": { >>> + "summary": "Python Tcl/Tk bindings", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/tkinter" >>> + ], >>> + "cached": [] >>> + }, >>> + "typing": { >>> + "summary": "Python typing support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/typing.py" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" >>> + ] >>> + }, >>> + "unittest": { >>> + "summary": "Python unit testing framework", >>> + "rdepends": [ >>> + "core", >>> + "difflib", >>> + "logging", >>> + "pprint", >>> + "shell", >>> + "stringold" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest", >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest/", >>> + "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" >>> + ], >>> + "cached": [] >>> + }, >>> + "unixadmin": { >>> + "summary": "Python Unix administration support", >>> + "rdepends": [ >>> + "core", >>> + "io" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/getpass.py", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" >>> + ], >>> + "cached": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" >>> + ] >>> + }, >>> + "venv": { >>> + "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", >>> + "rdepends": [ >>> + "compression", >>> + "core", >>> + "logging", >>> + "shell", >>> + "stringold", >>> + "unixadmin" >>> + ], >>> + "files": [ >>> + "${bindir}/pyvenv*", >>> + "${libdir}/python${PYTHON_MAJMIN}/venv" >>> + ], >>> + "cached": [] >>> + }, >>> + "xml": { >>> + "summary": "Python basic XML support", >>> + "rdepends": [ >>> + "core" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", >>> + "${libdir}/python${PYTHON_MAJMIN}/xml" >>> + ], >>> + "cached": [] >>> + }, >>> + "xmlrpc": { >>> + "summary": "Python XML-RPC support", >>> + "rdepends": [ >>> + "core", >>> + "xml" >>> + ], >>> + "files": [ >>> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", >>> + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" >>> + ], >>> + "cached": [] >>> + } >>> +} >>> diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest >>> new file mode 100644 >>> index 00000000000..3863c6d314f >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest >>> @@ -0,0 +1,3 @@ >>> +#!/bin/sh >>> + >>> +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' >>> diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >>> new file mode 100644 >>> index 00000000000..61fa12ffe8d >>> --- /dev/null >>> +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb >>> @@ -0,0 +1,281 @@ >>> +SUMMARY = "The Python Programming Language" >>> +HOMEPAGE ="http://www.python.org" >>> +LICENSE = "PSFv2" >>> +SECTION = "devel/python" >>> + >>> +LIC_FILES_CHKSUM ="file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754" >>> + >>> +SRC_URI ="http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ + >>> file://run-ptest \ + file://create_manifest3.py \ + >>> file://get_module_deps3.py \ + file://python3-manifest.json \ + >>> file://check_build_completeness.py \ + file://cgi_py.patch \ + >>> file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \ >>> + ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', >>> 'file://avoid_warning_about_tkinter.patch', d)} \ + >>> file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \ >>> + file://python-config.patch \ + >>> file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \ >>> + " >>> + >>> +SRC_URI_append_class-native = " \ >>> +file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ >>> +file://12-distutils-prefix-is-inside-staging-area.patch \ >>> + " >>> + >>> +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb" >>> +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb" >>> + >>> +# exclude pre-releases for both python 2.x and 3.x >>> +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" >>> + >>> +CVE_PRODUCT = "python" >>> + >>> +PYTHON_MAJMIN = "3.7" >>> +PYTHON_BINABI = "${PYTHON_MAJMIN}m" >>> + >>> +S = "${WORKDIR}/Python-${PV}" >>> + >>> +BBCLASSEXTEND = "native nativesdk" >>> + >>> +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives >>> + >>> +MULTILIB_SUFFIX ="${@d.getVar('base_libdir',1).split('/')[-1]}" >>> + >>> +ALTERNATIVE_${PN}-dev = "python-config" >>> +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" >>> +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" >>> + >>> + >>> +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2" >>> +DEPENDS_append_class-target = " python3-native" >>> +DEPENDS_append_class-nativesdk = " python3-native" >>> + >>> +EXTRA_OECONF = " --without-ensurepip --enable-shared" >>> +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}" >>> + >>> + >>> +EXTRANATIVEPATH += "python3-native" >>> + >>> +CACHED_CONFIGUREVARS = " \ >>> + ac_cv_file__dev_ptmx=yes \ >>> + ac_cv_file__dev_ptc=no \ >>> +" >>> + >>> +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" >>> +PACKAGECONFIG_class-native ??= "readline" >>> +PACKAGECONFIG_class-nativesdk ??= "readline" >>> +PACKAGECONFIG[readline] = ",,readline" >>> +# Use profile guided optimisation by running PyBench inside qemu-user >>> +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native" >>> +PACKAGECONFIG[tk] = ",,tk" >>> + >>> +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid" >>> + >>> +EXTRA_OEMAKE = '\ >>> + STAGING_LIBDIR=${STAGING_LIBDIR} \ >>> + STAGING_INCDIR=${STAGING_INCDIR} \ >>> +' >>> + >>> +do_compile_prepend_class-target() { >>> + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then >>> + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" >>> + cat >pgo-wrapper <<EOF >>> +#!/bin/sh >>> +cd ${B} >>> +$qemu_binary "\$@" >>> +EOF >>> + chmod +x pgo-wrapper >>> + fi >>> +} >>> + >>> +do_install_prepend() { >>> + ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile >>> +} >>> + >>> +do_install_append_class-target() { >>> + oe_multilib_header python${PYTHON_BINABI}/pyconfig.h >>> +} >>> + >>> +do_install_append_class-native() { >>> + # Make sure we use /usr/bin/env python >>> + for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do >>> + sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT >>> + done >>> + # Add a symlink to the native Python so that scripts can just invoke >>> + # "nativepython" and get the right one without needing absolute paths >>> + # (these often end up too long for the #! parser in the kernel as the >>> + # buffer is 128 bytes long). >>> + ln -s python3-native/python3 ${D}${bindir}/nativepython3 >>> +} >>> + >>> +do_install_append() { >>> + mkdir -p ${D}${libdir}/python-sysconfigdata >>> + sysconfigfile=`find ${D} -name _sysconfig*.py` >>> + cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >>> + >>> + sed -i \ >>> + -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \ >>> + -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >>> + -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ >>> + ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py >>> +} >>> + >>> +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py" >>> +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" >>> + >>> +py_package_preprocess () { >>> + # Remove references to buildmachine paths in target Makefile and _sysconfigdata >>> + sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ >>> + -e 's|${DEBUG_PREFIX_MAP}||g' \ >>> + -e 's:${HOSTTOOLS_DIR}/::g' \ >>> + -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ >>> + -e 's:${RECIPE_SYSROOT}::g' \ >>> + -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ >>> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \ >>> + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \ >>> + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config >>> + >>> + # Recompile _sysconfigdata after modifying it >>> + cd ${PKGD} >>> + sysconfigfile=`find . -name _sysconfigdata_*.py` >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile')" >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)" >>> + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ >>> + -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)" >>> + cd - >>> + >>> + mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} >>> + >>> + #Remove the unneeded copy of target sysconfig data >>> + rm -rf ${PKGD}/${libdir}/python-sysconfigdata >>> +} >>> + >>> +# We want bytecode precompiled .py files (.pyc's) by default >>> +# but the user may set it on their own conf >>> +INCLUDE_PYCS ?= "1" >>> + >>> +python(){ >>> + import collections, json >>> + >>> + filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') >>> + # This python changes the datastore based on the contents of a file, so mark >>> + # that dependency. >>> + bb.parse.mark_dependency(d, filename) >>> + >>> + with open(filename) as manifest_file: >>> + manifest_str = manifest_file.read() >>> + json_start = manifest_str.find('# EOC') + 6 >>> + manifest_file.seek(json_start) >>> + manifest_str = manifest_file.read() >>> + python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) >>> + >>> + # First set RPROVIDES for -native case >>> + # Hardcoded since it cant be python3-native-foo, should be python3-foo-native >>> + pn = 'python3' >>> + rprovides = d.getVar('RPROVIDES').split() >>> + >>> + for key in python_manifest: >>> + pypackage = pn + '-' + key + '-native' >>> + if pypackage not in rprovides: >>> + rprovides.append(pypackage) >>> + >>> + d.setVar('RPROVIDES_class-native', ' '.join(rprovides)) >>> + >>> + # Then work on the target >>> + include_pycs = d.getVar('INCLUDE_PYCS') >>> + >>> + packages = d.getVar('PACKAGES').split() >>> + pn = d.getVar('PN') >>> + >>> + newpackages=[] >>> + for key in python_manifest: >>> + pypackage= pn + '-' + key >>> + >>> + if pypackage not in packages: >>> + # We need to prepend, otherwise python-misc gets everything >>> + # so we use a new variable >>> + newpackages.append(pypackage) >>> + >>> + # "Build" python's manifest FILES, RDEPENDS and SUMMARY >>> + d.setVar('FILES_' + pypackage, '') >>> + for value in python_manifest[key]['files']: >>> + d.appendVar('FILES_' + pypackage, ' ' + value) >>> + >>> + # Add cached files >>> + if include_pycs == '1': >>> + for value in python_manifest[key]['cached']: >>> + d.appendVar('FILES_' + pypackage, ' ' + value) >>> + >>> + for value in python_manifest[key]['rdepends']: >>> + # Make it work with or without $PN >>> + if '${PN}' in value: >>> + value=value.split('-')[1] >>> + d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) >>> + d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) >>> + >>> + # Prepending so to avoid python-misc getting everything >>> + packages = newpackages + packages >>> + d.setVar('PACKAGES', ' '.join(packages)) >>> + d.setVar('ALLOW_EMPTY_${PN}-modules', '1') >>> +} >>> + >>> +# Files needed to create a new manifest >>> + >>> +do_create_manifest() { >>> + # This task should be run with every new release of Python. >>> + # We must ensure that PACKAGECONFIG enables everything when creating >>> + # a new manifest, this is to base our new manifest on a complete >>> + # native python build, containing all dependencies, otherwise the task >>> + # wont be able to find the required files. >>> + # e.g. BerkeleyDB is an optional build dependency so it may or may not >>> + # be present, we must ensure it is. >>> + >>> + cd ${WORKDIR} >>> + # This needs to be executed by python-native and NOT by HOST's python >>> + nativepython3 create_manifest3.py ${PYTHON_MAJMIN} >>> + cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json >>> +} >>> + >>> +# bitbake python -c create_manifest >>> +addtask do_create_manifest >>> + >>> +# Make sure we have native python ready when we create a new manifest >>> +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" >>> +do_create_manifest[depends] += "python3:do_patch" >>> + >>> +# manual dependency additions >>> +RPROVIDES_${PN}-modules = "${PN}" >>> +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" >>> +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" >>> + >>> +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" >>> +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" >>> +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" >>> + >>> +# provide python-pyvenv from python3-venv >>> +RPROVIDES_${PN}-venv += "python3-pyvenv" >>> + >>> +# package libpython3 >>> +PACKAGES =+ "libpython3 libpython3-staticdev" >>> +FILES_libpython3 = "${libdir}/libpython*.so.*" >>> +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a" >>> +INSANE_SKIP_${PN}-dev += "dev-elf" >>> + >>> +# catch all the rest (unsorted) >>> +PACKAGES += "${PN}-misc" >>> +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs" >>> +RDEPENDS_${PN}-modules += "python3-misc" >>> +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" >>> + >>> +# catch manpage >>> +PACKAGES += "${PN}-man" >>> +FILES_${PN}-man = "${datadir}/man" >>> + >>> +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" >>> +RDEPENDS_${PN}-tkinter +="${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" >>> +RDEPENDS_${PN}-dev = "" >>> + >>> -- >>> 2.17.1 >>> >>> -- >>> _______________________________________________ >>> Openembedded-core mailing list >>> Openembedded-core@lists.openembedded.org >>> http://lists.openembedded.org/mailman/listinfo/openembedded-core > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] python3: upgrade to 3.7.2 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego 2019-02-01 22:35 ` Alejandro Enedino Hernandez Samaniego 2019-02-03 2:08 ` Yu, Mingli @ 2019-02-04 15:01 ` Alexander Kanavin 2 siblings, 0 replies; 8+ messages in thread From: Alexander Kanavin @ 2019-02-04 15:01 UTC (permalink / raw) To: Alejandro Enedino Hernandez Samaniego Cc: Patches and discussions about the oe-core layer Regarding python-sanity directory, I simply wanted to place the new recipe somewhere entirely separate from the old recipe, so that I could refer to the old one and its patches easily, while making sure things don't get mixed up. Also, I think there *is* more sanity in the new recipe :) Alex On Fri, 1 Feb 2019 at 23:32, Alejandro Enedino Hernandez Samaniego <alejandro.enedino.hernandez-samaniego@xilinx.com> wrote: > > On 2/1/19 12:49 PM, Khem Raj wrote: > > On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > I took the same approach as the recent perl upgrade: write recipe from scratch, > taking the pieces from the old recipe only when they were proven to be necessary. > > The pgo, manifest and ptest features are all preserved. > > New features: > > - native and target recipes are now unified into one recipe > > - check_build_completeness.py runs right after do_compile() and verifies that > all optional modules have been built (a notorious source of regressions) > > - a new approach to sysconfig.py and distutils/sysconfig.py returning values > appropriate for native or target builds: we copy the configuration file to a > separate folder, add that folder to sys.path (through environment variable > that differs between native and target builds), and point python to the file > through another environment variable. > > Thanks, once current cycle is over for OE, I would like to give this a > whirl and see how it goes > > Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > --- > meta/classes/python3-dir.bbclass | 2 +- > meta/classes/python3native.bbclass | 2 + > ...ib-termcap-to-linker-flags-to-avoid-.patch | 25 + > ...hell-version-of-python-config-that-w.patch | 35 + > ...-qemu-wrapper-when-gathering-profile.patch | 25 + > ...fig-append-STAGING_LIBDIR-python-sys.patch | 42 + > ...tutils-prefix-is-inside-staging-area.patch | 54 + > .../python3/avoid_warning_about_tkinter.patch | 36 + > .../python-sanity/python3/cgi_py.patch | 32 + > .../python3/check_build_completeness.py | 17 + > .../python-sanity/python3/create_manifest3.py | 433 ++++++ > .../python-sanity/python3/get_module_deps3.py | 146 ++ > .../python-sanity/python3/python-config.patch | 46 + > .../python3/python3-manifest.json | 1227 +++++++++++++++++ > .../python-sanity/python3/run-ptest | 3 + > .../python-sanity/python3_3.7.2.bb | 281 ++++ > 16 files changed, 2405 insertions(+), 1 deletion(-) > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch > create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch > create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json > create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest > create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb > > Thanks Alex!, I looked at this briefly and it looks good. > > Do we want to keep the recipe at recipes-devtools-python-sanity?, also I think the PACKAGECONFIG for tk is broken but thats probably tk's fault, > and, since tk is part of meta-oe only, I am wondering if it would make sense to have a bbappend there with the PACKAGECONFIG there, > since we're depending on something that were actually not holding? > > Alejandro > > > > diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass > index 06bb046d9c2..7dd130bad99 100644 > --- a/meta/classes/python3-dir.bbclass > +++ b/meta/classes/python3-dir.bbclass > @@ -1,4 +1,4 @@ > -PYTHON_BASEVERSION = "3.5" > +PYTHON_BASEVERSION = "3.7" > PYTHON_ABI = "m" > PYTHON_DIR = "python${PYTHON_BASEVERSION}" > PYTHON_PN = "python3" > diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass > index da12a714703..a3acaf61bbd 100644 > --- a/meta/classes/python3native.bbclass > +++ b/meta/classes/python3native.bbclass > @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native " > export STAGING_INCDIR > export STAGING_LIBDIR > > +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" > + > # suppress host user's site-packages dirs. > export PYTHONNOUSERSITE = "1" > > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > new file mode 100644 > index 00000000000..09f279ba1d7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch > @@ -0,0 +1,25 @@ > +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Fri, 25 Jan 2019 19:04:13 +0100 > +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host > + contamination > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > + > +--- > + setup.py | 1 - > + 1 file changed, 1 deletion(-) > + > +diff --git a/setup.py b/setup.py > +index b4357e3..fbec00d 100644 > +--- a/setup.py > ++++ b/setup.py > +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext): > + 'termcap'): > + readline_libs.append('termcap') > + exts.append( Extension('readline', ['readline.c'], > +- library_dirs=['/usr/lib/termcap'], > + extra_link_args=readline_extra_link_args, > + libraries=readline_libs) ) > + else: > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > new file mode 100644 > index 00000000000..83fd52d87f4 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch > @@ -0,0 +1,35 @@ > +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Tue, 29 Jan 2019 15:03:01 +0100 > +Subject: [PATCH] Do not use the shell version of python-config that was > + introduced in 3.4 > + > +Revert instead to the original python version: it has our tweaks and > +outputs directories correctly. > + > +Upstream-Status: Inappropriate [oe-specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > +--- > + Makefile.pre.in | 9 +++------ > + 1 file changed, 3 insertions(+), 6 deletions(-) > + > +diff --git a/Makefile.pre.in b/Makefile.pre.in > +index 2d2e11f..cc19942 100644 > +--- a/Makefile.pre.in > ++++ b/Makefile.pre.in > +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh > + sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py > + @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR} > + LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config > +- @ # On Darwin, always use the python version of the script, the shell > +- @ # version doesn't use the compiler customizations that are provided > +- @ # in python (_osx_support.py). > +- @if test `uname -s` = Darwin; then \ > +- cp python-config.py python-config; \ > +- fi > ++ @ # In OpenEmbedded, always use the python version of the script, the shell > ++ @ # version is broken in multiple ways, and doesn't return correct directories > ++ cp python-config.py python-config > + > + > + # Install the include files > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > new file mode 100644 > index 00000000000..fa7735ff93e > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch > @@ -0,0 +1,25 @@ > +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Wed, 30 Jan 2019 12:41:04 +0100 > +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > +--- > + Makefile.pre.in | 3 +-- > + 1 file changed, 1 insertion(+), 2 deletions(-) > + > +diff --git a/Makefile.pre.in b/Makefile.pre.in > +index a3a02a7..d5503dd 100644 > +--- a/Makefile.pre.in > ++++ b/Makefile.pre.in > +@@ -507,8 +507,7 @@ build_all_generate_profile: > + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" > + > + run_profile_task: > +- @ # FIXME: can't run for a cross build > +- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true > ++ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true > + > + build_all_merge_profile: > + $(LLVM_PROF_MERGER) > diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > new file mode 100644 > index 00000000000..2c5b76f131c > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch > @@ -0,0 +1,42 @@ > +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex.kanavin@gmail.com> > +Date: Thu, 31 Jan 2019 16:46:30 +0100 > +Subject: [PATCH] distutils/sysconfig: append > + STAGING_LIBDIR/python-sysconfigdata to sys.path > + > +So that target configuration can be used when running native python > + > +Upstream-Status: Inappropriate [oe-core specific] > +Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> > + > +--- > + Lib/distutils/sysconfig.py | 2 ++ > + Lib/sysconfig.py | 2 ++ > + 2 files changed, 4 insertions(+) > + > +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py > +index e07a6c8..6b8c129 100644 > +--- a/Lib/distutils/sysconfig.py > ++++ b/Lib/distutils/sysconfig.py > +@@ -421,6 +421,8 @@ def _init_posix(): > + platform=sys.platform, > + multiarch=getattr(sys.implementation, '_multiarch', ''), > + )) > ++ if 'STAGING_LIBDIR' in os.environ: > ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') > + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > + build_time_vars = _temp.build_time_vars > + global _config_vars > +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py > +index 9ee4d31..e586abd 100644 > +--- a/Lib/sysconfig.py > ++++ b/Lib/sysconfig.py > +@@ -412,6 +412,8 @@ def _init_posix(vars): > + """Initialize the module as appropriate for POSIX systems.""" > + # _sysconfigdata is generated at build time, see _generate_posix_vars() > + name = _get_sysconfigdata_name() > ++ if 'STAGING_LIBDIR' in os.environ: > ++ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') > + _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > + build_time_vars = _temp.build_time_vars > + vars.update(build_time_vars) > diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > new file mode 100644 > index 00000000000..abf08f59dd7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch > @@ -0,0 +1,54 @@ > +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001 > +From: Khem Raj <raj.khem@gmail.com> > +Date: Tue, 14 May 2013 15:00:26 -0700 > +Subject: [PATCH] python3: Add target and native recipes > + > +Upstream-Status: Inappropriate [embedded specific] > + > +02/2015 Rebased for Python 3.4.2 > + > +# The proper prefix is inside our staging area. > +# Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de> > +# Signed-off-by: Phil Blundell <philb@gnu.org> > +# Signed-off-by: Khem Raj <raj.khem@gmail.com> > +# Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> > + > +--- > + Lib/distutils/sysconfig.py | 10 ++++++++-- > + 1 file changed, 8 insertions(+), 2 deletions(-) > + > +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py > +index 6b8c129..3ca7f79 100644 > +--- a/Lib/distutils/sysconfig.py > ++++ b/Lib/distutils/sysconfig.py > +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None): > + If 'prefix' is supplied, use it instead of sys.base_prefix or > + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. > + """ > +- if prefix is None: > ++ if prefix is None and os.environ['STAGING_INCDIR'] != "": > ++ prefix = os.environ['STAGING_INCDIR'].rstrip('include') > ++ elif prefix is None: > + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX > + if os.name == "posix": > + if python_build: > +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): > + If 'prefix' is supplied, use it instead of sys.base_prefix or > + sys.base_exec_prefix -- i.e., ignore 'plat_specific'. > + """ > ++ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1] > ++ if prefix is None and os.environ['STAGING_LIBDIR'] != "": > ++ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename) > ++ > + if prefix is None: > + if standard_lib: > + prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX > +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): > + > + if os.name == "posix": > + libpython = os.path.join(prefix, > +- "lib", "python" + get_python_version()) > ++ lib_basename, "python" + get_python_version()) > + if standard_lib: > + return libpython > + else: > diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > new file mode 100644 > index 00000000000..24e67b4ca14 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch > @@ -0,0 +1,36 @@ > +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001 > +From: Andrei Gherzan <andrei@gherzan.ro> > +Date: Mon, 28 Jan 2019 15:57:54 +0000 > +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet > + integrated in yocto so we skip the check for this module. Avoid a warning by > + not adding this module to missing variable. > + > +Upstream-Status: Inappropriate [distribution] > + > +Also simply disable the tk module since its not in DEPENDS. > +Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> > + > +--- > + setup.py | 8 +++++--- > + 1 file changed, 5 insertions(+), 3 deletions(-) > + > +diff --git a/setup.py b/setup.py > +index fbec00d..b7a36a6 100644 > +--- a/setup.py > ++++ b/setup.py > +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext): > + self.extensions.extend(exts) > + > + # Call the method for detecting whether _tkinter can be compiled > +- self.detect_tkinter(inc_dirs, lib_dirs) > ++ # self.detect_tkinter(inc_dirs, lib_dirs) > + > +- if '_tkinter' not in [e.name for e in self.extensions]: > +- missing.append('_tkinter') > ++ # tkinter module will not be avalaible as yocto > ++ # doesn't have tk integrated (yet) > ++ #if '_tkinter' not in [e.name for e in self.extensions]: > ++ # missing.append('_tkinter') > + > + # Build the _uuid module if possible > + uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"]) > diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch > new file mode 100644 > index 00000000000..6c4ba54320b > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch > @@ -0,0 +1,32 @@ > +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001 > +From: Mark Hatle <mark.hatle@windriver.com> > +Date: Wed, 21 Sep 2011 20:55:33 -0500 > +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment > + > +Upstream-Status: Inappropriate [distribution] > + > +Signed-off-by: Mark Hatle <mark.hatle@windriver.com> > + > +--- > + Lib/cgi.py | 11 +---------- > + 1 file changed, 1 insertion(+), 10 deletions(-) > + > +diff --git a/Lib/cgi.py b/Lib/cgi.py > +index 8cf6687..094c7b4 100755 > +--- a/Lib/cgi.py > ++++ b/Lib/cgi.py > +@@ -1,13 +1,4 @@ > +-#! /usr/local/bin/python > +- > +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is > +-# intentionally NOT "/usr/bin/env python". On many systems > +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI > +-# scripts, and /usr/local/bin is the default directory where Python is > +-# installed, so /usr/bin/env would be unable to find python. Granted, > +-# binary installations by Linux vendors often install Python in > +-# /usr/bin. So let those vendors patch cgi.py to match their choice > +-# of installation. > ++#! /usr/bin/env python > + > + """Support module for CGI (Common Gateway Interface) scripts. > + > diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > new file mode 100755 > index 00000000000..a1eace3f571 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py > @@ -0,0 +1,17 @@ > +#!/usr/bin/env python3 > +import sys > +logfile = open(sys.argv[1]).read() > + > +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found") > +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.") > +if necessary_bits != -1: > + print("%s" %(logfile[necessary_bits:to_find_bits])) > + > +failed_to_build = logfile.find("Failed to build these modules:") > +if failed_to_build != -1: > + failed_to_build_end = logfile.find("\n\n", failed_to_build) > + print("%s" %(logfile[failed_to_build:failed_to_build_end])) > + > +if necessary_bits != -1 or failed_to_build != -1: > + sys.exit(1) > + > diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py > new file mode 100644 > index 00000000000..4da02a2991a > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py > @@ -0,0 +1,433 @@ > +# This script is used as a bitbake task to create a new python manifest > +# $ bitbake python -c create_manifest > +# > +# Our goal is to keep python-core as small as posible and add other python > +# packages only when the user needs them, hence why we split upstream python > +# into several packages. > +# > +# In a very simplistic way what this does is: > +# Launch python and see specifically what is required for it to run at a minimum > +# > +# Go through the python-manifest file and launch a separate task for every single > +# one of the files on each package, this task will check what was required for that > +# specific module to run, these modules will be called dependencies. > +# The output of such task will be a list of the modules or dependencies that were > +# found for that file. > +# > +# Such output will be parsed by this script, we will look for each dependency on the > +# manifest and if we find that another package already includes it, then we will add > +# that package as an RDEPENDS to the package we are currently checking; in case we dont > +# find the current dependency on any other package we will add it to the current package > +# as part of FILES. > +# > +# > +# This way we will create a new manifest from the data structure that was built during > +# this process, on this new manifest each package will contain specifically only > +# what it needs to run. > +# > +# There are some caveats which we try to deal with, such as repeated files on different > +# packages, packages that include folders, wildcards, and special packages. > +# Its also important to note that this method only works for python files, and shared > +# libraries. Static libraries, header files and binaries need to be dealt with manually. > +# > +# This script differs from its python2 version mostly on how shared libraries are handled > +# The manifest file for python3 has an extra field which contains the cached files for > +# each package. > +# Tha method to handle cached files does not work when a module includes a folder which > +# itself contains the pycache folder, gladly this is almost never the case. > +# > +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com> > + > + > +import sys > +import subprocess > +import json > +import os > +import collections > + > +# Get python version from ${PYTHON_MAJMIN} > +pyversion = str(sys.argv[1]) > + > +# Hack to get native python search path (for folders), not fond of it but it works for now > +pivot = 'recipe-sysroot-native' > +for p in sys.path: > + if pivot in p: > + nativelibfolder = p[:p.find(pivot)+len(pivot)] > + > +# Empty dict to hold the whole manifest > +new_manifest = collections.OrderedDict() > + > +# Check for repeated files, folders and wildcards > +allfiles = [] > +repeated = [] > +wildcards = [] > + > +hasfolders = [] > +allfolders = [] > + > +def isFolder(value): > + value = value.replace('${PYTHON_MAJMIN}',pyversion) > + if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')): > + return True > + else: > + return False > + > +def isCached(item): > + if '__pycache__' in item: > + return True > + else: > + return False > + > +def prepend_comments(comments, json_manifest): > + with open(json_manifest, 'r+') as manifest: > + json_contents = manifest.read() > + manifest.seek(0, 0) > + manifest.write(comments + json_contents) > + > +# Read existing JSON manifest > +with open('python3-manifest.json') as manifest: > + # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker > + manifest_str = manifest.read() > + json_start = manifest_str.find('# EOC') + 6 # EOC + \n > + manifest.seek(0) > + comments = manifest.read(json_start) > + manifest_str = manifest.read() > + old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) > + > +# > +# First pass to get core-package functionality, because we base everything on the fact that core is actually working > +# Not exactly the same so it should not be a function > +# > + > +print ('Getting dependencies for package: core') > + > + > +# This special call gets the core dependencies and > +# appends to the old manifest so it doesnt hurt what it > +# currently holds. > +# This way when other packages check for dependencies > +# on the new core package, they will still find them > +# even when checking the old_manifest > + > +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8') > +for coredep in output.split(): > + coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}') > + if isCached(coredep): > + if coredep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(coredep) > + else: > + if coredep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(coredep) > + > + > +# The second step is to loop through the existing files contained in the core package > +# according to the old manifest, identify if they are modules, or some other type > +# of file that we cant import (directories, binaries, configs) in which case we > +# can only assume they were added correctly (manually) so we ignore those and > +# pass them to the manifest directly. > + > +for filedep in old_manifest['core']['files']: > + if isFolder(filedep): > + if isCached(filedep): > + if filedep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(filedep) > + else: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + if '${bindir}' in filedep: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + if filedep == '': > + continue > + if '${includedir}' in filedep: > + if filedep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(filedep) > + continue > + > + # Get actual module name , shouldnt be affected by libdir/bindir, etc. > + pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0] > + > + > + # We now know that were dealing with a python module, so we can import it > + # and check what its dependencies are. > + # We launch a separate task for each module for deterministic behavior. > + # Each module will only import what is necessary for it to work in specific. > + # The output of each task will contain each module's dependencies > + > + print ('Getting dependencies for module: %s' % pymodule) > + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') > + print ('The following dependencies were found for module %s:\n' % pymodule) > + print (output) > + > + > + for pymodule_dep in output.split(): > + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') > + > + if isCached(pymodule_dep): > + if pymodule_dep not in old_manifest['core']['cached']: > + old_manifest['core']['cached'].append(pymodule_dep) > + else: > + if pymodule_dep not in old_manifest['core']['files']: > + old_manifest['core']['files'].append(pymodule_dep) > + > + > +# At this point we are done with the core package. > +# The old_manifest dictionary is updated only for the core package because > +# all others will use this a base. > + > + > +# To improve the script speed, we check which packages contain directories > +# since we will be looping through (only) those later. > +for pypkg in old_manifest: > + for filedep in old_manifest[pypkg]['files']: > + if isFolder(filedep): > + print ('%s is a folder' % filedep) > + if pypkg not in hasfolders: > + hasfolders.append(pypkg) > + if filedep not in allfolders: > + allfolders.append(filedep) > + > + > + > +# This is the main loop that will handle each package. > +# It works in a similar fashion than the step before, but > +# we will now be updating a new dictionary that will eventually > +# become the new manifest. > +# > +# The following loops though all packages in the manifest, > +# through all files on each of them, and checks whether or not > +# they are modules and can be imported. > +# If they can be imported, then it checks for dependencies for > +# each of them by launching a separate task. > +# The output of that task is then parsed and the manifest is updated > +# accordingly, wether it should add the module on FILES for the current package > +# or if that module already belongs to another package then the current one > +# will RDEPEND on it > + > +for pypkg in old_manifest: > + # Use an empty dict as data structure to hold data for each package and fill it up > + new_manifest[pypkg] = collections.OrderedDict() > + new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary'] > + new_manifest[pypkg]['rdepends'] = [] > + new_manifest[pypkg]['files'] = [] > + new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached'] > + > + # All packages should depend on core > + if pypkg != 'core': > + new_manifest[pypkg]['rdepends'].append('core') > + new_manifest[pypkg]['cached'] = [] > + > + print('\n') > + print('--------------------------') > + print ('Handling package %s' % pypkg) > + print('--------------------------') > + > + # Handle special cases, we assume that when they were manually added > + # to the manifest we knew what we were doing. > + special_packages = ['misc', 'modules', 'dev', 'tests'] > + if pypkg in special_packages or 'staticdev' in pypkg: > + print('Passing %s package directly' % pypkg) > + new_manifest[pypkg] = old_manifest[pypkg] > + continue > + > + for filedep in old_manifest[pypkg]['files']: > + # We already handled core on the first pass, we can ignore it now > + if pypkg == 'core': > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + > + # Handle/ignore what we cant import > + if isFolder(filedep): > + new_manifest[pypkg]['files'].append(filedep) > + # Asyncio (and others) are both the package and the folder name, we should not skip those... > + path,mod = os.path.split(filedep) > + if mod != pypkg: > + continue > + if '${bindir}' in filedep: > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + if filedep == '': > + continue > + if '${includedir}' in filedep: > + if filedep not in new_manifest[pypkg]['files']: > + new_manifest[pypkg]['files'].append(filedep) > + continue > + > + # Get actual module name , shouldnt be affected by libdir/bindir, etc. > + # We need to check if the imported module comes from another (e.g. sqlite3.dump) > + path,pymodule = os.path.split(filedep) > + path = os.path.basename(path) > + pymodule = os.path.splitext(os.path.basename(pymodule))[0] > + > + # If this condition is met, it means we need to import it from another module > + # or its the folder itself (e.g. unittest) > + if path == pypkg: > + if pymodule: > + pymodule = path + '.' + pymodule > + else: > + pymodule = path > + > + > + > + # We now know that were dealing with a python module, so we can import it > + # and check what its dependencies are. > + # We launch a separate task for each module for deterministic behavior. > + # Each module will only import what is necessary for it to work in specific. > + # The output of each task will contain each module's dependencies > + > + print ('\nGetting dependencies for module: %s' % pymodule) > + output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8') > + print ('The following dependencies were found for module %s:\n' % pymodule) > + print (output) > + > + reportFILES = [] > + reportRDEPS = [] > + > + for pymodule_dep in output.split(): > + > + # Warning: This first part is ugly > + # One of the dependencies that was found, could be inside of one of the folders included by another package > + # We need to check if this happens so we can add the package containing the folder as an rdependency > + # e.g. Folder encodings contained in codecs > + # This would be solved if no packages included any folders > + > + # This can be done in two ways: > + # 1 - We assume that if we take out the filename from the path we would get > + # the folder string, then we would check if folder string is in the list of folders > + # This would not work if a package contains a folder which contains another folder > + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 > + # folder_string would not match any value contained in the list of folders > + # > + # 2 - We do it the other way around, checking if the folder is contained in the path > + # e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2 > + # is folder_string inside path/folder1/folder2/filename?, > + # Yes, it works, but we waste a couple of milliseconds. > + > + pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}') > + inFolders = False > + for folder in allfolders: > + # The module could have a directory named after it, e.g. xml, if we take out the filename from the path > + # we'll end up with ${libdir}, and we want ${libdir}/xml > + if isFolder(pymodule_dep): > + check_path = pymodule_dep > + else: > + check_path = os.path.dirname(pymodule_dep) > + if folder in check_path : > + inFolders = True # Did we find a folder? > + folderFound = False # Second flag to break inner for > + # Loop only through packages which contain folders > + for pypkg_with_folder in hasfolders: > + if (folderFound == False): > + # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder)) > + for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']: > + if folder_dep == folder: > + print ('%s folder found in %s' % (folder, pypkg_with_folder)) > + folderFound = True > + if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg: > + new_manifest[pypkg]['rdepends'].append(pypkg_with_folder) > + else: > + break > + > + # A folder was found so we're done with this item, we can go on > + if inFolders: > + continue > + > + > + > + # No directories beyond this point > + # We might already have this module on the dictionary since it could depend on a (previously checked) module > + if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']: > + # Handle core as a special package, we already did it so we pass it to NEW data structure directly > + if pypkg == 'core': > + print('Adding %s to %s FILES' % (pymodule_dep, pypkg)) > + if pymodule_dep.endswith('*'): > + wildcards.append(pymodule_dep) > + if isCached(pymodule_dep): > + new_manifest[pypkg]['cached'].append(pymodule_dep) > + else: > + new_manifest[pypkg]['files'].append(pymodule_dep) > + > + # Check for repeated files > + if pymodule_dep not in allfiles: > + allfiles.append(pymodule_dep) > + else: > + if pymodule_dep not in repeated: > + repeated.append(pymodule_dep) > + else: > + > + > + # Last step: Figure out if we this belongs to FILES or RDEPENDS > + # We check if this module is already contained on another package, so we add that one > + # as an RDEPENDS, or if its not, it means it should be contained on the current > + # package, and we should add it to FILES > + for possible_rdep in old_manifest: > + # Debug > + # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep) > + if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']: > + # Since were nesting, we need to check its not the same pypkg > + if(possible_rdep != pypkg): > + if possible_rdep not in new_manifest[pypkg]['rdepends']: > + # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs > + reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep)) > + new_manifest[pypkg]['rdepends'].append(possible_rdep) > + break > + else: > + > + # Since this module wasnt found on another package, it is not an RDEP, > + # so we add it to FILES for this package. > + # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files) > + if os.path.basename(pymodule_dep) != pypkg: > + reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg))) > + if isCached(pymodule_dep): > + new_manifest[pypkg]['cached'].append(pymodule_dep) > + else: > + new_manifest[pypkg]['files'].append(pymodule_dep) > + if pymodule_dep.endswith('*'): > + wildcards.append(pymodule_dep) > + if pymodule_dep not in allfiles: > + allfiles.append(pymodule_dep) > + else: > + if pymodule_dep not in repeated: > + repeated.append(pymodule_dep) > + > + print('\n') > + print('#################################') > + print('Summary for module %s' % pymodule) > + print('FILES found for module %s:' % pymodule) > + print(''.join(reportFILES)) > + print('RDEPENDS found for module %s:' % pymodule) > + print(''.join(reportRDEPS)) > + print('#################################') > + > +print('The following FILES contain wildcards, please check if they are necessary') > +print(wildcards) > +print('The following FILES contain folders, please check if they are necessary') > +print(hasfolders) > + > + > +# Sort it just so it looks nicer > +for pypkg in new_manifest: > + new_manifest[pypkg]['files'].sort() > + new_manifest[pypkg]['cached'].sort() > + new_manifest[pypkg]['rdepends'].sort() > + > +# Create the manifest from the data structure that was built > +with open('python3-manifest.json.new','w') as outfile: > + json.dump(new_manifest,outfile, indent=4) > + outfile.write('\n') > + > +prepend_comments(comments,'python3-manifest.json.new') > + > +if (repeated): > + error_msg = '\n\nERROR:\n' > + error_msg += 'The following files are repeated (contained in more than one package),\n' > + error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n' > + error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n' > + error_msg += '\n'.join(repeated) > + error_msg += '\n' > + sys.exit(error_msg) > + > diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > new file mode 100644 > index 00000000000..fd12baad84e > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py > @@ -0,0 +1,146 @@ > +# This script is launched on separate task for each python module > +# It checks for dependencies for that specific module and prints > +# them out, the output of this execution will have all dependencies > +# for a specific module, which will be parsed an dealt on create_manifest.py > +# > +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com> > + > +# We can get a log per module, for all the dependencies that were found, but its messy. > +debug=False > + > +import sys > + > +# We can get a list of the modules which are currently required to run python > +# so we run python-core and get its modules, we then import what we need > +# and check what modules are currently running, if we substract them from the > +# modules we had initially, we get the dependencies for the module we imported. > + > +# We use importlib to achieve this, so we also need to know what modules importlib needs > +import importlib > + > +core_deps=set(sys.modules) > + > +def fix_path(dep_path): > + import os > + # We DONT want the path on our HOST system > + pivot='recipe-sysroot-native' > + dep_path=dep_path[dep_path.find(pivot)+len(pivot):] > + > + if '/usr/bin' in dep_path: > + dep_path = dep_path.replace('/usr/bin''${bindir}') > + > + # Handle multilib, is there a better way? > + if '/usr/lib32' in dep_path: > + dep_path = dep_path.replace('/usr/lib32','${libdir}') > + if '/usr/lib64' in dep_path: > + dep_path = dep_path.replace('/usr/lib64','${libdir}') > + if '/usr/lib' in dep_path: > + dep_path = dep_path.replace('/usr/lib','${libdir}') > + if '/usr/include' in dep_path: > + dep_path = dep_path.replace('/usr/include','${includedir}') > + if '__init__.' in dep_path: > + dep_path = os.path.split(dep_path)[0] > + return dep_path > + > + > +# Module to import was passed as an argument > +current_module = str(sys.argv[1]).rstrip() > +if(debug==True): > + log = open('log_%s' % current_module,'w') > + log.write('Module %s generated the following dependencies:\n' % current_module) > +try: > + importlib.import_module('%s' % current_module) > +except ImportError as e: > + if (debug==True): > + log.write('Module was not found') > + pass > + > + > +# Get current module dependencies, dif will contain a list of specific deps for this module > +module_deps=set(sys.modules) > + > +# We handle the core package (1st pass on create_manifest.py) as a special case > +if current_module == 'python-core-package': > + dif = core_deps > +else: > + # We know this is not the core package, so there must be a difference. > + dif = module_deps-core_deps > + > + > +# Check where each dependency came from > +for item in dif: > + dep_path='' > + try: > + if (debug==True): > + log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n') > + dep_path = sys.modules['%s' % item].__file__ > + except AttributeError as e: > + # Deals with thread (builtin module) not having __file__ attribute > + if debug==True: > + log.write(item + ' ') > + log.write(str(e)) > + log.write('\n') > + pass > + except NameError as e: > + # Deals with NameError: name 'dep_path' is not defined > + # because module is not found (wasn't compiled?), e.g. bddsm > + if (debug==True): > + log.write(item+' ') > + log.write(str(e)) > + pass > + > + # Site-customize is a special case since we (OpenEmbedded) put it there manually > + if 'sitecustomize' in dep_path: > + dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py' > + # Prints out result, which is what will be used by create_manifest > + print (dep_path) > + continue > + > + dep_path = fix_path(dep_path) > + > + import sysconfig > + soabi=sysconfig.get_config_var('SOABI') > + # Check if its a shared library and deconstruct it > + if soabi in dep_path: > + if (debug==True): > + log.write('Shared library found in %s' % dep_path) > + dep_path = dep_path.replace(soabi,'*') > + print (dep_path) > + continue > + > + if (debug==True): > + log.write(dep_path+'\n') > + # Prints out result, which is what will be used by create_manifest > + print (dep_path) > + > + > + import imp > + cpython_tag = imp.get_tag() > + cached='' > + # Theres no naive way to find *.pyc files on python3 > + try: > + if (debug==True): > + log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n') > + cached = sys.modules['%s' % item].__cached__ > + except AttributeError as e: > + # Deals with thread (builtin module) not having __cached__ attribute > + if debug==True: > + log.write(item + ' ') > + log.write(str(e)) > + log.write('\n') > + pass > + except NameError as e: > + # Deals with NameError: name 'cached' is not defined > + if (debug==True): > + log.write(item+' ') > + log.write(str(e)) > + pass > + if cached is not None: > + if (debug==True): > + log.write(cached) > + cached = fix_path(cached) > + cached = cached.replace(cpython_tag,'*') > + print (cached) > + > +if debug==True: > + log.close() > diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch > new file mode 100644 > index 00000000000..f23b8b7df06 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch > @@ -0,0 +1,46 @@ > +python-config: Revert to using distutils.sysconfig > + > +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in > + > +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig > + > +affect the native runtime as well as cross building. Use the old, patched > +implementation which returns paths in the staging directory and for the target, > +as appropriate. > + > +Upstream-Status: Inappropriate [Embedded Specific] > + > +Signed-off-by: Tyler Hall <tylerwhall@gmail.com> > +: > +Index: Python-3.3.3/Misc/python-config.in > +=================================================================== > +--- Python-3.3.3.orig/Misc/python-config.in > ++++ Python-3.3.3/Misc/python-config.in > +@@ -4,7 +4,7 @@ > + import getopt > + import os > + import sys > +-import sysconfig > ++from distutils import sysconfig > + > + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', > + 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir'] > +@@ -32,14 +32,14 @@ if '--help' in opt_flags: > + > + for opt in opt_flags: > + if opt == '--prefix': > +- print(sysconfig.get_config_var('prefix')) > ++ print(sysconfig.PREFIX) > + > + elif opt == '--exec-prefix': > +- print(sysconfig.get_config_var('exec_prefix')) > ++ print(sysconfig.EXEC_PREFIX) > + > + elif opt in ('--includes', '--cflags'): > +- flags = ['-I' + sysconfig.get_path('include'), > +- '-I' + sysconfig.get_path('platinclude')] > ++ flags = ['-I' + sysconfig.get_python_inc(), > ++ '-I' + sysconfig.get_python_inc(plat_specific=True)] > + if opt == '--cflags': > + flags.extend(getvar('CFLAGS').split()) > + print(' '.join(flags)) > diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json > new file mode 100644 > index 00000000000..82c7075f0e7 > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json > @@ -0,0 +1,1227 @@ > +# DO NOT (entirely) modify this file manually, please read. > +# > +# IMPORTANT NOTE: > +# Please keep in mind that the create_manifest task relies on the fact the the > +# target and native Python packages are the same, and it also needs to be executed > +# with a fully working native package (with all the PACKAGECONFIGs enabled and all > +# and all the modules should be working, check log.do_compile), otherwise the script > +# will fail to find dependencies correctly, this note is valid either if you are > +# upgrading to a new Python version or adding a new package. > +# > +# > +# If you are adding a new package please follow the next steps: > +# How to add a new package: > +# - If a user wants to add a new package all that has to be done is: > +# Modify the python3-manifest.json file, and add the required file(s) to the FILES list, > +# fill up the SUMMARY section as well, the script should handle all the rest. > +# > +# Real example: > +# We want to add a web browser package, including the file webbrowser.py > +# which at the moment is on python3-misc. > +# "webbrowser": { > +# "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"], > +# "rdepends": [], > +# "summary": "Python Web Browser support"} > +# > +# * Note that the rdepends field was left empty > +# > +# We run $ bitbake python3 -c create_manifest and the resulting manifest > +# should be completed after a few seconds, showing something like: > +# "webbrowser": { > +# "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"], > +# "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], > +# "summary": "Python Web Browser support"} > +# > +# > +# If you are upgrading Python to a new version please follow the next steps: > +# After each Python upgrade, the create_manifest task should be executed, because we > +# don't control what changes on upstream Python, so, some module dependency > +# might have changed without us realizing it, a certain module can either have > +# more or less dependencies, or could be depending on a new file that was just > +# created on the new release and for obvious reasons we wouldn't have it on our > +# old manifest, all of these issues would cause runtime errors on our system. > +# > +# - Upgrade both the native and target Python packages to a new version > +# - Run the create_manifest task for the target Python package as its shown below: > +# > +# $ bitbake python3 -c create_manifest > +# > +# This will automatically replace your manifest file located under the Python directory > +# with an new one, which contains the new dependencies (if any). > +# > +# Several things could have gone wrong here, I will try to explain a few: > +# > +# a) A new file was introduced on this release, e.g. sha3*.so: > +# The task will check what its needed to import every module, more than one module would > +# would probably depend on sha3*.so, although only one module should contain it. > +# > +# After running the task, the new manifest will have the sha3*.so file on more than one > +# module, you need to manually decide which one of them should get it and delete it from > +# the others, for example sha3*.so should likely be on ${PN}-crypt. > +# Once you have deleted from the others you need to run the create_manifest task again, > +# this will populate the other module's rdepends fields, with ${PN}-crypt and you should be > +# good to go. > +# > +# b) The native package wasn't built correctly and its missing a certain module: > +# As mentioned before, you need to make sure the native package was built with all the modules > +# because it is used as base to build the manifest file, you need to manually check log.do_compile > +# since it won't error out the compile function if its only missing a couple of modules. > +# > +# e.g. missing the _uuid module, log.do_compile would show the following: > +# Python build finished successfully! > +# The necessary bits to build these optional modules were not found: > +# _uuid > +# > +# What will happen here is that the new manifest would not be aware that the _uuid module exists, so > +# not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on > +# the misc package (which is where any file that doesn't belong anywhere else ends up). > +# > +# This will eventually cause runtime errors on our system if we don't include the misc package on > +# on our image, because the _uuid files will be missing. > +# If we build the _uuid module correctly and run the create_manifest task the _uuid files will be > +# detected correctly along with its dependencies, and we will get a working manifest. > +# > +# This is the reason why it is important to make sure we have a fully working native build, > +# so we can avoid these errors. > +# > +# > +# > +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST > +# EOC > +{ > + "tests": { > + "summary": "Python test suite", > + "rdepends": [ > + "core", > + "modules" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/*/test", > + "${libdir}/python${PYTHON_MAJMIN}/*/tests", > + "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/", > + "${libdir}/python${PYTHON_MAJMIN}/test" > + ], > + "cached": [] > + }, > + "2to3": { > + "summary": "Python automated Python 2 to 3 code translator", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${bindir}/2to3-*", > + "${libdir}/python${PYTHON_MAJMIN}/lib2to3" > + ], > + "cached": [] > + }, > + "asyncio": { > + "summary": "Python Asynchronous I/", > + "rdepends": [ > + "core", > + "io", > + "logging", > + "netclient", > + "numbers", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/asyncio", > + "${libdir}/python${PYTHON_MAJMIN}/concurrent", > + "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so" > + ], > + "cached": [] > + }, > + "audio": { > + "summary": "Python Audio Handling", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/chunk.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py", > + "${libdir}/python${PYTHON_MAJMIN}/sunau.py", > + "${libdir}/python${PYTHON_MAJMIN}/wave.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc" > + ] > + }, > + "codecs": { > + "summary": "Python codec", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc" > + ] > + }, > + "compile": { > + "summary": "Python bytecode compilation support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/compileall.py", > + "${libdir}/python${PYTHON_MAJMIN}/py_compile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc" > + ] > + }, > + "compression": { > + "summary": "Python high-level compression support", > + "rdepends": [ > + "core", > + "shell", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_compression.py", > + "${libdir}/python${PYTHON_MAJMIN}/bz2.py", > + "${libdir}/python${PYTHON_MAJMIN}/gzip.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lzma.py", > + "${libdir}/python${PYTHON_MAJMIN}/tarfile.py", > + "${libdir}/python${PYTHON_MAJMIN}/zipfile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc" > + ] > + }, > + "core": { > + "summary": "Python interpreter and core modules", > + "rdepends": [], > + "files": [ > + "${bindir}/python*[!-config]", > + "${includedir}/python${PYTHON_BINABI}/pyconfig*.h", > + "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", > + "${libdir}/python${PYTHON_MAJMIN}/UserList.py", > + "${libdir}/python${PYTHON_MAJMIN}/UserString.py", > + "${libdir}/python${PYTHON_MAJMIN}/__future__.py", > + "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py", > + "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py", > + "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py", > + "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py", > + "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py", > + "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py", > + "${libdir}/python${PYTHON_MAJMIN}/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/argparse.py", > + "${libdir}/python${PYTHON_MAJMIN}/ast.py", > + "${libdir}/python${PYTHON_MAJMIN}/bisect.py", > + "${libdir}/python${PYTHON_MAJMIN}/code.py", > + "${libdir}/python${PYTHON_MAJMIN}/codecs.py", > + "${libdir}/python${PYTHON_MAJMIN}/codeop.py", > + "${libdir}/python${PYTHON_MAJMIN}/collections", > + "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/configparser.py", > + "${libdir}/python${PYTHON_MAJMIN}/contextlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/copy.py", > + "${libdir}/python${PYTHON_MAJMIN}/copyreg.py", > + "${libdir}/python${PYTHON_MAJMIN}/csv.py", > + "${libdir}/python${PYTHON_MAJMIN}/dis.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py", > + "${libdir}/python${PYTHON_MAJMIN}/enum.py", > + "${libdir}/python${PYTHON_MAJMIN}/functools.py", > + "${libdir}/python${PYTHON_MAJMIN}/genericpath.py", > + "${libdir}/python${PYTHON_MAJMIN}/getopt.py", > + "${libdir}/python${PYTHON_MAJMIN}/gettext.py", > + "${libdir}/python${PYTHON_MAJMIN}/heapq.py", > + "${libdir}/python${PYTHON_MAJMIN}/imp.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py", > + "${libdir}/python${PYTHON_MAJMIN}/inspect.py", > + "${libdir}/python${PYTHON_MAJMIN}/io.py", > + "${libdir}/python${PYTHON_MAJMIN}/keyword.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/linecache.py", > + "${libdir}/python${PYTHON_MAJMIN}/locale.py", > + "${libdir}/python${PYTHON_MAJMIN}/new.py", > + "${libdir}/python${PYTHON_MAJMIN}/opcode.py", > + "${libdir}/python${PYTHON_MAJMIN}/operator.py", > + "${libdir}/python${PYTHON_MAJMIN}/optparse.py", > + "${libdir}/python${PYTHON_MAJMIN}/os.py", > + "${libdir}/python${PYTHON_MAJMIN}/platform.py", > + "${libdir}/python${PYTHON_MAJMIN}/posixpath.py", > + "${libdir}/python${PYTHON_MAJMIN}/re.py", > + "${libdir}/python${PYTHON_MAJMIN}/reprlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py", > + "${libdir}/python${PYTHON_MAJMIN}/selectors.py", > + "${libdir}/python${PYTHON_MAJMIN}/signal.py", > + "${libdir}/python${PYTHON_MAJMIN}/site.py", > + "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py", > + "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py", > + "${libdir}/python${PYTHON_MAJMIN}/stat.py", > + "${libdir}/python${PYTHON_MAJMIN}/stringprep.py", > + "${libdir}/python${PYTHON_MAJMIN}/struct.py", > + "${libdir}/python${PYTHON_MAJMIN}/subprocess.py", > + "${libdir}/python${PYTHON_MAJMIN}/symbol.py", > + "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py", > + "${libdir}/python${PYTHON_MAJMIN}/textwrap.py", > + "${libdir}/python${PYTHON_MAJMIN}/threading.py", > + "${libdir}/python${PYTHON_MAJMIN}/token.py", > + "${libdir}/python${PYTHON_MAJMIN}/tokenize.py", > + "${libdir}/python${PYTHON_MAJMIN}/traceback.py", > + "${libdir}/python${PYTHON_MAJMIN}/types.py", > + "${libdir}/python${PYTHON_MAJMIN}/warnings.py", > + "${libdir}/python${PYTHON_MAJMIN}/weakref.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc" > + ] > + }, > + "crypt": { > + "summary": "Python basic cryptographic and hashing support", > + "rdepends": [ > + "core", > + "math", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/crypt.py", > + "${libdir}/python${PYTHON_MAJMIN}/hashlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc" > + ] > + }, > + "ctypes": { > + "summary": "Python C types support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/ctypes", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so" > + ], > + "cached": [] > + }, > + "curses": { > + "summary": "Python curses support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/curses", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so" > + ], > + "cached": [] > + }, > + "datetime": { > + "summary": "Python calendar and time support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_strptime.py", > + "${libdir}/python${PYTHON_MAJMIN}/calendar.py", > + "${libdir}/python${PYTHON_MAJMIN}/datetime.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc" > + ] > + }, > + "db": { > + "summary": "Python file-based database support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/dbm", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so" > + ], > + "cached": [] > + }, > + "debugger": { > + "summary": "Python debugger", > + "rdepends": [ > + "core", > + "pprint", > + "shell", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/bdb.py", > + "${libdir}/python${PYTHON_MAJMIN}/pdb.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc" > + ] > + }, > + "dev": { > + "cached": [], > + "files": [ > + "${base_libdir}/*.a", > + "${base_libdir}/*.o", > + "${bindir}/python*-config", > + "${datadir}/aclocal", > + "${datadir}/pkgconfig", > + "${includedir}", > + "${libdir}/*.a", > + "${libdir}/*.la", > + "${libdir}/*.o", > + "${libdir}/lib*${SOLIBSDEV}", > + "${libdir}/pkgconfig", > + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile", > + "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__" > + ], > + "rdepends": [ > + "core" > + ], > + "summary": "Python development package" > + }, > + "difflib": { > + "summary": "Python helpers for computing deltas between objects", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/difflib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc" > + ] > + }, > + "distutils-staticdev": { > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a" > + ], > + "rdepends": [ > + "distutils" > + ], > + "summary": "Python distribution utilities (static libraries)" > + }, > + "distutils": { > + "summary": "Python Distribution Utilities", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/distutils" > + ], > + "cached": [] > + }, > + "doctest": { > + "summary": "Python framework for running examples in docstrings", > + "rdepends": [ > + "core", > + "debugger", > + "difflib", > + "logging", > + "pprint", > + "shell", > + "stringold", > + "unittest" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/doctest.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc" > + ] > + }, > + "email": { > + "summary": "Python email support", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "io", > + "math", > + "netclient" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/email", > + "${libdir}/python${PYTHON_MAJMIN}/imaplib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc" > + ] > + }, > + "fcntl": { > + "summary": "Python's fcntl interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so" > + ], > + "cached": [] > + }, > + "gdbm": { > + "summary": "Python GNU database support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so" > + ], > + "cached": [] > + }, > + "html": { > + "summary": "Python HTML processing support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/formatter.py", > + "${libdir}/python${PYTHON_MAJMIN}/html" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc" > + ] > + }, > + "idle": { > + "summary": "Python Integrated Development Environment", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${bindir}/idle*", > + "${libdir}/python${PYTHON_MAJMIN}/idlelib" > + ], > + "cached": [] > + }, > + "image": { > + "summary": "Python graphical image handling", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/colorsys.py", > + "${libdir}/python${PYTHON_MAJMIN}/imghdr.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc" > + ] > + }, > + "io": { > + "summary": "Python low-level I/O", > + "rdepends": [ > + "compression", > + "core", > + "crypt", > + "math", > + "netclient", > + "shell", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_pyio.py", > + "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/pipes.py", > + "${libdir}/python${PYTHON_MAJMIN}/socket.py", > + "${libdir}/python${PYTHON_MAJMIN}/ssl.py", > + "${libdir}/python${PYTHON_MAJMIN}/tempfile.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc" > + ] > + }, > + "json": { > + "summary": "Python JSON support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/json", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so" > + ], > + "cached": [] > + }, > + "logging": { > + "summary": "Python logging support", > + "rdepends": [ > + "core", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/logging" > + ], > + "cached": [] > + }, > + "mailbox": { > + "summary": "Python mailbox format support", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "fcntl", > + "io", > + "math", > + "mime", > + "netclient", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/mailbox.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc" > + ] > + }, > + "math": { > + "summary": "Python math support", > + "rdepends": [ > + "core", > + "crypt" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/random.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc" > + ] > + }, > + "mime": { > + "summary": "Python MIME handling APIs", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/quopri.py", > + "${libdir}/python${PYTHON_MAJMIN}/uu.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc" > + ] > + }, > + "mmap": { > + "summary": "Python memory-mapped file support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so" > + ], > + "cached": [] > + }, > + "modules": { > + "cached": [], > + "files": [], > + "rdepends": [ > + "2to3", > + "asyncio", > + "audio", > + "codecs", > + "compile", > + "compression", > + "core", > + "crypt", > + "ctypes", > + "curses", > + "datetime", > + "db", > + "debugger", > + "difflib", > + "distutils", > + "doctest", > + "email", > + "fcntl", > + "html", > + "idle", > + "image", > + "io", > + "json", > + "logging", > + "mailbox", > + "math", > + "mime", > + "mmap", > + "multiprocessing", > + "netclient", > + "netserver", > + "numbers", > + "pickle", > + "pkgutil", > + "plistlib", > + "pprint", > + "profile", > + "pydoc", > + "resource", > + "runpy", > + "shell", > + "smtpd", > + "sqlite3", > + "stringold", > + "syslog", > + "terminal", > + "threading", > + "tkinter", > + "typing", > + "unittest", > + "unixadmin", > + "venv", > + "xml", > + "xmlrpc" > + ], > + "summary": "All Python modules" > + }, > + "multiprocessing": { > + "summary": "Python multiprocessing support", > + "rdepends": [ > + "core", > + "io", > + "pickle" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/multiprocessing" > + ], > + "cached": [] > + }, > + "netclient": { > + "summary": "Python Internet Protocol clients", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "io", > + "math", > + "mime", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/base64.py", > + "${libdir}/python${PYTHON_MAJMIN}/ftplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/hmac.py", > + "${libdir}/python${PYTHON_MAJMIN}/http", > + "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py", > + "${libdir}/python${PYTHON_MAJMIN}/nntplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/poplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/smtplib.py", > + "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py", > + "${libdir}/python${PYTHON_MAJMIN}/urllib", > + "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__", > + "${libdir}/python${PYTHON_MAJMIN}/uuid.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc" > + ] > + }, > + "netserver": { > + "summary": "Python Internet Protocol servers", > + "rdepends": [ > + "compression", > + "core", > + "crypt", > + "datetime", > + "email", > + "html", > + "io", > + "math", > + "mime", > + "netclient", > + "shell", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cgi.py", > + "${libdir}/python${PYTHON_MAJMIN}/socketserver.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc" > + ] > + }, > + "numbers": { > + "summary": "Python number APIs", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py", > + "${libdir}/python${PYTHON_MAJMIN}/contextvars.py", > + "${libdir}/python${PYTHON_MAJMIN}/decimal.py", > + "${libdir}/python${PYTHON_MAJMIN}/fractions.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/numbers.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc" > + ] > + }, > + "pickle": { > + "summary": "Python serialisation/persistence support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/pickle.py", > + "${libdir}/python${PYTHON_MAJMIN}/pickletools.py", > + "${libdir}/python${PYTHON_MAJMIN}/shelve.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc" > + ] > + }, > + "pkgutil": { > + "summary": "Python package extension utility support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc" > + ] > + }, > + "plistlib": { > + "summary": "Generate and parse Mac OS X .plist files", > + "rdepends": [ > + "core", > + "datetime", > + "xml" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/plistlib.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc" > + ] > + }, > + "pprint": { > + "summary": "Python pretty-print support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pprint.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc" > + ] > + }, > + "profile": { > + "summary": "Python basic performance profiling support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cProfile.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/profile.py", > + "${libdir}/python${PYTHON_MAJMIN}/pstats.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc" > + ] > + }, > + "pydoc": { > + "summary": "Python interactive help support", > + "rdepends": [ > + "core", > + "netclient", > + "pkgutil" > + ], > + "files": [ > + "${bindir}/pydoc*", > + "${libdir}/python${PYTHON_MAJMIN}/pydoc.py", > + "${libdir}/python${PYTHON_MAJMIN}/pydoc_data" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc" > + ] > + }, > + "resource": { > + "summary": "Python resource control interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so" > + ], > + "cached": [] > + }, > + "runpy": { > + "summary": "Python helper for locating/executing scripts in module namespace", > + "rdepends": [ > + "core", > + "pkgutil" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/runpy.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc" > + ] > + }, > + "shell": { > + "summary": "Python shell-like functionality", > + "rdepends": [ > + "compression", > + "core", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/cmd.py", > + "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py", > + "${libdir}/python${PYTHON_MAJMIN}/glob.py", > + "${libdir}/python${PYTHON_MAJMIN}/shlex.py", > + "${libdir}/python${PYTHON_MAJMIN}/shutil.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc" > + ] > + }, > + "smtpd": { > + "summary": "Python Simple Mail Transport Daemon", > + "rdepends": [ > + "core", > + "crypt", > + "datetime", > + "email", > + "io", > + "math", > + "mime", > + "netclient", > + "stringold" > + ], > + "files": [ > + "${bindir}/smtpd.py", > + "${libdir}/python${PYTHON_MAJMIN}/asynchat.py", > + "${libdir}/python${PYTHON_MAJMIN}/asyncore.py", > + "${libdir}/python${PYTHON_MAJMIN}/smtpd.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc" > + ] > + }, > + "sqlite3": { > + "summary": "Python Sqlite3 database support", > + "rdepends": [ > + "core", > + "datetime" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/sqlite3" > + ], > + "cached": [] > + }, > + "stringold": { > + "summary": "Python string APIs [deprecated]", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/string.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc" > + ] > + }, > + "syslog": { > + "summary": "Python syslog interface", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so" > + ], > + "cached": [] > + }, > + "terminal": { > + "summary": "Python terminal controlling support", > + "rdepends": [ > + "core", > + "io" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/pty.py", > + "${libdir}/python${PYTHON_MAJMIN}/tty.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc" > + ] > + }, > + "threading": { > + "summary": "Python threading & synchronization support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py", > + "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/queue.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc", > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc" > + ] > + }, > + "tkinter": { > + "summary": "Python Tcl/Tk bindings", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/tkinter" > + ], > + "cached": [] > + }, > + "typing": { > + "summary": "Python typing support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/typing.py" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc" > + ] > + }, > + "unittest": { > + "summary": "Python unit testing framework", > + "rdepends": [ > + "core", > + "difflib", > + "logging", > + "pprint", > + "shell", > + "stringold" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/unittest", > + "${libdir}/python${PYTHON_MAJMIN}/unittest/", > + "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__" > + ], > + "cached": [] > + }, > + "unixadmin": { > + "summary": "Python Unix administration support", > + "rdepends": [ > + "core", > + "io" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/getpass.py", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so" > + ], > + "cached": [ > + "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc" > + ] > + }, > + "venv": { > + "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.", > + "rdepends": [ > + "compression", > + "core", > + "logging", > + "shell", > + "stringold", > + "unixadmin" > + ], > + "files": [ > + "${bindir}/pyvenv*", > + "${libdir}/python${PYTHON_MAJMIN}/venv" > + ], > + "cached": [] > + }, > + "xml": { > + "summary": "Python basic XML support", > + "rdepends": [ > + "core" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so", > + "${libdir}/python${PYTHON_MAJMIN}/xml" > + ], > + "cached": [] > + }, > + "xmlrpc": { > + "summary": "Python XML-RPC support", > + "rdepends": [ > + "core", > + "xml" > + ], > + "files": [ > + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc", > + "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__" > + ], > + "cached": [] > + } > +} > diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest > new file mode 100644 > index 00000000000..3863c6d314f > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest > @@ -0,0 +1,3 @@ > +#!/bin/sh > + > +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g' > diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb > new file mode 100644 > index 00000000000..61fa12ffe8d > --- /dev/null > +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb > @@ -0,0 +1,281 @@ > +SUMMARY = "The Python Programming Language" > +HOMEPAGE = "http://www.python.org" > +LICENSE = "PSFv2" > +SECTION = "devel/python" > + > +LIC_FILES_CHKSUM = "file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754" > + > +SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ > + file://run-ptest \ > + file://create_manifest3.py \ > + file://get_module_deps3.py \ > + file://python3-manifest.json \ > + file://check_build_completeness.py \ > + file://cgi_py.patch \ > + file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \ > + ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \ > + file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \ > + file://python-config.patch \ > + file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \ > + " > + > +SRC_URI_append_class-native = " \ > + file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ > + file://12-distutils-prefix-is-inside-staging-area.patch \ > + " > + > +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb" > +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb" > + > +# exclude pre-releases for both python 2.x and 3.x > +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar" > + > +CVE_PRODUCT = "python" > + > +PYTHON_MAJMIN = "3.7" > +PYTHON_BINABI = "${PYTHON_MAJMIN}m" > + > +S = "${WORKDIR}/Python-${PV}" > + > +BBCLASSEXTEND = "native nativesdk" > + > +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives > + > +MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}" > + > +ALTERNATIVE_${PN}-dev = "python-config" > +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config" > +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}" > + > + > +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2" > +DEPENDS_append_class-target = " python3-native" > +DEPENDS_append_class-nativesdk = " python3-native" > + > +EXTRA_OECONF = " --without-ensurepip --enable-shared" > +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}" > + > + > +EXTRANATIVEPATH += "python3-native" > + > +CACHED_CONFIGUREVARS = " \ > + ac_cv_file__dev_ptmx=yes \ > + ac_cv_file__dev_ptc=no \ > +" > + > +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" > +PACKAGECONFIG_class-native ??= "readline" > +PACKAGECONFIG_class-nativesdk ??= "readline" > +PACKAGECONFIG[readline] = ",,readline" > +# Use profile guided optimisation by running PyBench inside qemu-user > +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native" > +PACKAGECONFIG[tk] = ",,tk" > + > +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid" > + > +EXTRA_OEMAKE = '\ > + STAGING_LIBDIR=${STAGING_LIBDIR} \ > + STAGING_INCDIR=${STAGING_INCDIR} \ > +' > + > +do_compile_prepend_class-target() { > + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then > + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" > + cat >pgo-wrapper <<EOF > +#!/bin/sh > +cd ${B} > +$qemu_binary "\$@" > +EOF > + chmod +x pgo-wrapper > + fi > +} > + > +do_install_prepend() { > + ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile > +} > + > +do_install_append_class-target() { > + oe_multilib_header python${PYTHON_BINABI}/pyconfig.h > +} > + > +do_install_append_class-native() { > + # Make sure we use /usr/bin/env python > + for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do > + sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT > + done > + # Add a symlink to the native Python so that scripts can just invoke > + # "nativepython" and get the right one without needing absolute paths > + # (these often end up too long for the #! parser in the kernel as the > + # buffer is 128 bytes long). > + ln -s python3-native/python3 ${D}${bindir}/nativepython3 > +} > + > +do_install_append() { > + mkdir -p ${D}${libdir}/python-sysconfigdata > + sysconfigfile=`find ${D} -name _sysconfig*.py` > + cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py > + > + sed -i \ > + -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \ > + -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ > + -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \ > + ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py > +} > + > +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py" > +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess" > + > +py_package_preprocess () { > + # Remove references to buildmachine paths in target Makefile and _sysconfigdata > + sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ > + -e 's|${DEBUG_PREFIX_MAP}||g' \ > + -e 's:${HOSTTOOLS_DIR}/::g' \ > + -e 's:${RECIPE_SYSROOT_NATIVE}::g' \ > + -e 's:${RECIPE_SYSROOT}::g' \ > + -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \ > + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \ > + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \ > + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config > + > + # Recompile _sysconfigdata after modifying it > + cd ${PKGD} > + sysconfigfile=`find . -name _sysconfigdata_*.py` > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile')" > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)" > + ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ > + -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)" > + cd - > + > + mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX} > + > + #Remove the unneeded copy of target sysconfig data > + rm -rf ${PKGD}/${libdir}/python-sysconfigdata > +} > + > +# We want bytecode precompiled .py files (.pyc's) by default > +# but the user may set it on their own conf > +INCLUDE_PYCS ?= "1" > + > +python(){ > + import collections, json > + > + filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json') > + # This python changes the datastore based on the contents of a file, so mark > + # that dependency. > + bb.parse.mark_dependency(d, filename) > + > + with open(filename) as manifest_file: > + manifest_str = manifest_file.read() > + json_start = manifest_str.find('# EOC') + 6 > + manifest_file.seek(json_start) > + manifest_str = manifest_file.read() > + python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict) > + > + # First set RPROVIDES for -native case > + # Hardcoded since it cant be python3-native-foo, should be python3-foo-native > + pn = 'python3' > + rprovides = d.getVar('RPROVIDES').split() > + > + for key in python_manifest: > + pypackage = pn + '-' + key + '-native' > + if pypackage not in rprovides: > + rprovides.append(pypackage) > + > + d.setVar('RPROVIDES_class-native', ' '.join(rprovides)) > + > + # Then work on the target > + include_pycs = d.getVar('INCLUDE_PYCS') > + > + packages = d.getVar('PACKAGES').split() > + pn = d.getVar('PN') > + > + newpackages=[] > + for key in python_manifest: > + pypackage= pn + '-' + key > + > + if pypackage not in packages: > + # We need to prepend, otherwise python-misc gets everything > + # so we use a new variable > + newpackages.append(pypackage) > + > + # "Build" python's manifest FILES, RDEPENDS and SUMMARY > + d.setVar('FILES_' + pypackage, '') > + for value in python_manifest[key]['files']: > + d.appendVar('FILES_' + pypackage, ' ' + value) > + > + # Add cached files > + if include_pycs == '1': > + for value in python_manifest[key]['cached']: > + d.appendVar('FILES_' + pypackage, ' ' + value) > + > + for value in python_manifest[key]['rdepends']: > + # Make it work with or without $PN > + if '${PN}' in value: > + value=value.split('-')[1] > + d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value) > + d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary']) > + > + # Prepending so to avoid python-misc getting everything > + packages = newpackages + packages > + d.setVar('PACKAGES', ' '.join(packages)) > + d.setVar('ALLOW_EMPTY_${PN}-modules', '1') > +} > + > +# Files needed to create a new manifest > + > +do_create_manifest() { > + # This task should be run with every new release of Python. > + # We must ensure that PACKAGECONFIG enables everything when creating > + # a new manifest, this is to base our new manifest on a complete > + # native python build, containing all dependencies, otherwise the task > + # wont be able to find the required files. > + # e.g. BerkeleyDB is an optional build dependency so it may or may not > + # be present, we must ensure it is. > + > + cd ${WORKDIR} > + # This needs to be executed by python-native and NOT by HOST's python > + nativepython3 create_manifest3.py ${PYTHON_MAJMIN} > + cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json > +} > + > +# bitbake python -c create_manifest > +addtask do_create_manifest > + > +# Make sure we have native python ready when we create a new manifest > +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot" > +do_create_manifest[depends] += "python3:do_patch" > + > +# manual dependency additions > +RPROVIDES_${PN}-modules = "${PN}" > +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules" > +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates" > + > +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}" > +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" > +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}" > + > +# provide python-pyvenv from python3-venv > +RPROVIDES_${PN}-venv += "python3-pyvenv" > + > +# package libpython3 > +PACKAGES =+ "libpython3 libpython3-staticdev" > +FILES_libpython3 = "${libdir}/libpython*.so.*" > +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a" > +INSANE_SKIP_${PN}-dev += "dev-elf" > + > +# catch all the rest (unsorted) > +PACKAGES += "${PN}-misc" > +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs" > +RDEPENDS_${PN}-modules += "python3-misc" > +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}" > + > +# catch manpage > +PACKAGES += "${PN}-man" > +FILES_${PN}-man = "${datadir}/man" > + > +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2" > +RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}" > +RDEPENDS_${PN}-dev = "" > + > -- > 2.17.1 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-02-04 15:01 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-02-01 12:58 [PATCH 1/3] python3: upgrade to 3.7.2 Alexander Kanavin 2019-02-01 12:58 ` [PATCH 2/3] python3: delete old 3.5.6 version Alexander Kanavin 2019-02-01 12:58 ` [PATCH 3/3] lib/oeqa/runtime/cases/gi.py: fix deprecation warning Alexander Kanavin 2019-02-01 20:49 ` [PATCH 1/3] python3: upgrade to 3.7.2 Khem Raj 2019-02-01 22:32 ` Alejandro Enedino Hernandez Samaniego 2019-02-01 22:35 ` Alejandro Enedino Hernandez Samaniego 2019-02-03 2:08 ` Yu, Mingli 2019-02-04 15:01 ` Alexander Kanavin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox