* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python @ 2017-07-20 7:33 Diankun Zhang 2017-07-20 7:56 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 7:33 UTC (permalink / raw) To: buildroot ? The python buildin module _hashlib has dependency on library libcrypto.so to provide the SHA-224, SHA-256, SHA-384, SHA-512 hash algorithms in addition to platform optimized versions of MD5 and SHA1. Adding host package dependency on host-openssl to support the building of _hashlib module for host-python. Signed-off-by: Diankun Zhang <kundy.zhang@gmail.com> --- package/python/python.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package/python/python.mk b/package/python/python.mk index cf1ddfb..248393c 100644 --- a/package/python/python.mk +++ b/package/python/python.mk @@ -56,7 +56,7 @@ HOST_PYTHON_MAKE = $(MAKE1) PYTHON_DEPENDENCIES = host-python libffi $(TARGET_NLS_DEPENDENCIES) -HOST_PYTHON_DEPENDENCIES = host-expat host-zlib +HOST_PYTHON_DEPENDENCIES = host-expat host-zlib host-openssl PYTHON_INSTALL_STAGING = YES -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/990c22a6/attachment.html> ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 7:33 [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python Diankun Zhang @ 2017-07-20 7:56 ` Thomas Petazzoni 2017-07-20 8:21 ` Diankun Zhang 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2017-07-20 7:56 UTC (permalink / raw) To: buildroot Hello, On Thu, 20 Jul 2017 15:33:17 +0800, Diankun Zhang wrote: > The python buildin module _hashlib has dependency on library > libcrypto.so to provide the SHA-224, SHA-256, SHA-384, SHA-512 > hash algorithms in addition to platform optimized versions > of MD5 and SHA1. Adding host package dependency on host-openssl > to support the building of _hashlib module for host-python. > > Signed-off-by: Diankun Zhang <kundy.zhang@gmail.com> Why do you need the _hashlib module in the host-python ? Indeed, we try to avoid adding too many dependencies, and adding a new dependency to host-python is going to increase the build time to a large number of people who don't necessarily need the _hashlib module in the host Python interpreter. So could you explain why this is needed ? Thanks! Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 7:56 ` Thomas Petazzoni @ 2017-07-20 8:21 ` Diankun Zhang 2017-07-20 8:27 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 8:21 UTC (permalink / raw) To: buildroot Hello Thomas, According to HOST_PYTHON_CONF_OPTS defined in python.mk, --disable-hashlib is not specified. That means _hashlib should be enabled by default, right? On Thu, Jul 20, 2017 at 3:56 PM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 20 Jul 2017 15:33:17 +0800, Diankun Zhang wrote: > > > The python buildin module _hashlib has dependency on library > > libcrypto.so to provide the SHA-224, SHA-256, SHA-384, SHA-512 > > hash algorithms in addition to platform optimized versions > > of MD5 and SHA1. Adding host package dependency on host-openssl > > to support the building of _hashlib module for host-python. > > > > Signed-off-by: Diankun Zhang <kundy.zhang@gmail.com> > > Why do you need the _hashlib module in the host-python ? > > Indeed, we try to avoid adding too many dependencies, and adding a new > dependency to host-python is going to increase the build time to a > large number of people who don't necessarily need the _hashlib module > in the host Python interpreter. > > So could you explain why this is needed ? > > Thanks! > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/5934de83/attachment.html> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 8:21 ` Diankun Zhang @ 2017-07-20 8:27 ` Thomas Petazzoni 2017-07-20 9:09 ` Diankun Zhang 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2017-07-20 8:27 UTC (permalink / raw) To: buildroot Hello, On Thu, 20 Jul 2017 16:21:51 +0800, Diankun Zhang wrote: > According to HOST_PYTHON_CONF_OPTS defined in python.mk, --disable-hashlib > is not specified. > That means _hashlib should be enabled by default, right? Well, no, not necessarily. We simply forgot to add --disable-hashlib. Again: why do you need this? If you could answer this question it would make the discussion a lot more productive :-) It is very possible that there are legitimate use cases for having the hashlib module built in the host Python interpreter. If we understand those use cases, then we can come up with a solution. But if you keep being silent about the *why* you need this, we won't move very far. Thanks! Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 8:27 ` Thomas Petazzoni @ 2017-07-20 9:09 ` Diankun Zhang 2017-07-20 9:19 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 9:09 UTC (permalink / raw) To: buildroot Hello Thomas, I'm cross-compiling host-python on my x86_64 box on which ubuntu 16.04 installed. libssl is installed whereas openssl is not installed on my box. When cross-compiling host-python, I found _hashlib.so is not compiled whereas _md5.so, _sha256.so, _sha512.so and _md5.so are compiled. I think -disable-hashlib will not decrease the build time and the size of the host-python artifacts. As for the hash lib use case, i am integrating a third-party python tool pyang (https://github.com/mbj4668/pyang) into our project. The pyang tool will take advantage of python _hashlib module. On Thu, Jul 20, 2017 at 4:27 PM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 20 Jul 2017 16:21:51 +0800, Diankun Zhang wrote: > > > According to HOST_PYTHON_CONF_OPTS defined in python.mk, > --disable-hashlib > > is not specified. > > That means _hashlib should be enabled by default, right? > > Well, no, not necessarily. We simply forgot to add --disable-hashlib. > > Again: why do you need this? If you could answer this question it would > make the discussion a lot more productive :-) > > It is very possible that there are legitimate use cases for having > the hashlib module built in the host Python interpreter. If we > understand those use cases, then we can come up with a solution. But if > you keep being silent about the *why* you need this, we won't move very > far. > > Thanks! > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/eef0bd84/attachment.html> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 9:09 ` Diankun Zhang @ 2017-07-20 9:19 ` Thomas Petazzoni 2017-07-20 10:59 ` Diankun Zhang 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2017-07-20 9:19 UTC (permalink / raw) To: buildroot Hello, On Thu, 20 Jul 2017 17:09:52 +0800, Diankun Zhang wrote: > I'm cross-compiling host-python on my x86_64 box on which ubuntu 16.04 You are not cross-compiling host-python. host-python is a host package, so it is natively compiled. > installed. > libssl is installed whereas openssl is not installed on my box. When In Ubuntu 16.04, the package is named libssl, but it's actually OpenSSL, see https://packages.ubuntu.com/xenial/libssl1.0.0. Its source package is "openssl". > cross-compiling > host-python, I found _hashlib.so is not compiled whereas _md5.so, > _sha256.so, _sha512.so > and _md5.so are compiled. I think -disable-hashlib will not decrease the > build time and > the size of the host-python artifacts. --disable-hashlib will make sure that our host-python build is more reproducible. Instead of having _hashlib built when libssl-dev is installed on the host machine, and _hashlib not built otherwise, it will consistently not build _hashlib, which is what we want. > As for the hash lib use case, i am integrating a third-party python tool > pyang > (https://github.com/mbj4668/pyang) into our project. The pyang tool will > take advantage of python > _hashlib module. It pyang meant to be run on the host or on the target ? From a quick look, it seems like something that runs on the host indeed. If that's the case, then we will really need to start adding hidden options for host packages, so that other host packages can request hashlib support from host-python, without forcing everyone to build this. Best regards, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 9:19 ` Thomas Petazzoni @ 2017-07-20 10:59 ` Diankun Zhang 2017-07-20 11:09 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 10:59 UTC (permalink / raw) To: buildroot Hello Thomas, From https://packages.ubuntu.com/xenial/amd64/openssl/filelist, we can see the openssl package on ubuntu 16.04 does not cotain the libssl or licrypto.so, instead they are in package libssl1.0.0, https://packages.ubuntu.com/xenial/amd64/libssl1.0.0/filelist. In most of systems nowadays, openssl (and libssl ) is installed since the hash algorithms are needed to meet kinds of requirements. Yes, I'm using the pyang tool on my host to compile some YANG files. The compiled YANG files are platform independent ideally. On Thu, Jul 20, 2017 at 5:19 PM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 20 Jul 2017 17:09:52 +0800, Diankun Zhang wrote: > > > I'm cross-compiling host-python on my x86_64 box on which ubuntu 16.04 > > You are not cross-compiling host-python. host-python is a host package, > so it is natively compiled. > > > installed. > > libssl is installed whereas openssl is not installed on my box. When > > In Ubuntu 16.04, the package is named libssl, but it's actually > OpenSSL, see https://packages.ubuntu.com/xenial/libssl1.0.0. Its source > package is "openssl". > > > cross-compiling > > host-python, I found _hashlib.so is not compiled whereas _md5.so, > > _sha256.so, _sha512.so > > and _md5.so are compiled. I think -disable-hashlib will not decrease the > > build time and > > the size of the host-python artifacts. > > --disable-hashlib will make sure that our host-python build is more > reproducible. Instead of having _hashlib built when libssl-dev is > installed on the host machine, and _hashlib not built otherwise, it > will consistently not build _hashlib, which is what we want. > > > As for the hash lib use case, i am integrating a third-party python tool > > pyang > > (https://github.com/mbj4668/pyang) into our project. The pyang tool will > > take advantage of python > > _hashlib module. > > It pyang meant to be run on the host or on the target ? From a quick > look, it seems like something that runs on the host indeed. If that's > the case, then we will really need to start adding hidden options for > host packages, so that other host packages can request hashlib support > from host-python, without forcing everyone to build this. > > Best regards, > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/7d436c9b/attachment.html> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 10:59 ` Diankun Zhang @ 2017-07-20 11:09 ` Thomas Petazzoni 2017-07-20 11:20 ` Diankun Zhang 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2017-07-20 11:09 UTC (permalink / raw) To: buildroot Hello, On Thu, 20 Jul 2017 18:59:31 +0800, Diankun Zhang wrote: > From https://packages.ubuntu.com/xenial/amd64/openssl/filelist, we can see > the openssl package on ubuntu 16.04 does not cotain the libssl or > licrypto.so, > instead they are in package libssl1.0.0, > https://packages.ubuntu.com/xenial/amd64/libssl1.0.0/filelist. libssl is a *binary* package that is produced from the source package openssl. > In most of systems nowadays, openssl (and libssl ) is installed since the > hash algorithms are needed to meet kinds of requirements. Yes, the libraries, but not necessarily the development files. > Yes, I'm using the pyang tool on my host to compile some YANG files. The > compiled YANG files are platform independent ideally. OK. So, again what is needed is a hidden BR2_PACKAGE_HOST_PYTHON_OPENSSL that enables OpenSSL support in host-python, which the pyang package could select. Best regards, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 11:09 ` Thomas Petazzoni @ 2017-07-20 11:20 ` Diankun Zhang 2017-07-20 11:24 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 11:20 UTC (permalink / raw) To: buildroot Thanks, Thomas, I would send out a new patch as you suggested. On Thu, Jul 20, 2017 at 7:09 PM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 20 Jul 2017 18:59:31 +0800, Diankun Zhang wrote: > > > From https://packages.ubuntu.com/xenial/amd64/openssl/filelist, we can > see > > the openssl package on ubuntu 16.04 does not cotain the libssl or > > licrypto.so, > > instead they are in package libssl1.0.0, > > https://packages.ubuntu.com/xenial/amd64/libssl1.0.0/filelist. > > libssl is a *binary* package that is produced from the source package > openssl. > > > In most of systems nowadays, openssl (and libssl ) is installed since the > > hash algorithms are needed to meet kinds of requirements. > > Yes, the libraries, but not necessarily the development files. > > > Yes, I'm using the pyang tool on my host to compile some YANG files. The > > compiled YANG files are platform independent ideally. > > OK. > > So, again what is needed is a hidden BR2_PACKAGE_HOST_PYTHON_OPENSSL > that enables OpenSSL support in host-python, which the pyang package > could select. > > Best regards, > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/34cb0ecf/attachment.html> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 11:20 ` Diankun Zhang @ 2017-07-20 11:24 ` Thomas Petazzoni 2017-07-20 11:39 ` Diankun Zhang 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2017-07-20 11:24 UTC (permalink / raw) To: buildroot Hello, On Thu, 20 Jul 2017 19:20:16 +0800, Diankun Zhang wrote: > Thanks, Thomas, I would send out a new patch as you suggested. But are you going to submit the patch that adds a package for pyang as well ? Without the pyang addition, it doesn't make a lot of sense to add support for building host-python with host-openssl. Best regards, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python 2017-07-20 11:24 ` Thomas Petazzoni @ 2017-07-20 11:39 ` Diankun Zhang 0 siblings, 0 replies; 11+ messages in thread From: Diankun Zhang @ 2017-07-20 11:39 UTC (permalink / raw) To: buildroot Thomas, will it be much better if I add hashlib package ( https://pypi.python.org/pypi/hashlib/20081119) into buildroot? On Thu, Jul 20, 2017 at 7:24 PM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 20 Jul 2017 19:20:16 +0800, Diankun Zhang wrote: > > > Thanks, Thomas, I would send out a new patch as you suggested. > > But are you going to submit the patch that adds a package for pyang as > well ? Without the pyang addition, it doesn't make a lot of sense to > add support for building host-python with host-openssl. > > Best regards, > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > -- -Diankun -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20170720/d9a49102/attachment.html> ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2017-07-20 11:39 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-07-20 7:33 [Buildroot] [PATCH 1/1] package/python: fix building _hashlib of host-python Diankun Zhang 2017-07-20 7:56 ` Thomas Petazzoni 2017-07-20 8:21 ` Diankun Zhang 2017-07-20 8:27 ` Thomas Petazzoni 2017-07-20 9:09 ` Diankun Zhang 2017-07-20 9:19 ` Thomas Petazzoni 2017-07-20 10:59 ` Diankun Zhang 2017-07-20 11:09 ` Thomas Petazzoni 2017-07-20 11:20 ` Diankun Zhang 2017-07-20 11:24 ` Thomas Petazzoni 2017-07-20 11:39 ` Diankun Zhang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox