[Buildroot] [Bug 7628] New: Python SSL does not get built for Raspberry Pi

[Buildroot] [Bug 7628] New: Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

           Summary: Python SSL does not get built for Raspberry Pi
           Product: buildroot
           Version: 2014.08
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: tobias.meggendorfer+buildroot at gmail.com
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


I'm building for the raspberry pi and want to have the Python SSL package.

Steps i've taken:
- make distclean
- make raspberrypi_defconfig
- make nconfig
- Enable WCHAR support, python3 and python-ssl
- make
- grab coffee

However, there is no "_ssl.cpython-34m.so" ("_ssl.so" in the python2 case) to
be found in the lib-dynload folder, thus the ssl module isn't working. I'm
fairly new to (cross-)compiling so I might have missed something trivial,
although I think I've doublechecked everything. I guess there just is a include
or something like that missing, as the python compilation complains about ssl
dependencies not being found.

I've also tried this with glibc 2.18 / 2.19 and gcc 4.8 / 4.9, binutils 2.22 /
2.24 and python2 and EABIhf. I actually need glibc + EABIhf for my application,
but this problem seems to occur with every configuration.

Host: 3.2.0-70-virtual #105-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
Buildroot: git version 2014.08-0-gfcd720d

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-11-13 23:02:54 UTC ---
Can you give your exact Buildroot .config? I've built the following defconfig
here:

BR2_arm=y
BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON3_SSL=y

And I definitely do have the _ss.cpython-34m.so library built:

# find / -name '_ssl*'
/usr/lib/python3.4/lib-dynload/_ssl.cpython-34m.so

However, when I try to use the ssl module, it complains of an undefined
instruction:

# python
Python 3.4.1 (default, Nov 13 2014, 22:55:45) 
[GCC 4.8.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
python (528): undefined instruction: pc=769e536c
Code: 00000000 f26ee1fe e12fff1e ee190f1d (e12fff1e) 
>>> 

So there is indeed an issue, but it does not appear to be the issue you
reported.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #2 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:14:02 UTC ---
Created attachment 5774
  --> https://bugs.busybox.net/attachment.cgi?id=5774
My config

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #3 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:21:11 UTC ---
I attached my config, highlighting the following:
BR2_arm=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_PACKAGE_PYTHON3_PYC_ONLY=y
BR2_PACKAGE_PYTHON3_SSL=y
BR2_PACKAGE_PYTHON3_UNICODEDATA=y

find output/target/ -name "_ssl*" (in the buildroot main directory) definitely
is empty with the given configuration (at least on my machine)

The getty port apparently differs but I don't think this causes any issues. I'm
currently building with the modified getty port and PYC_ONLY=n, just in case,
will edit results in here.

I don't know how well the env is cleaned when compiling, so here is the output
of env:
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=<cookie>
SSH_CLIENT=<ip & port>
SSH_TTY=/dev/pts/1
USER=<username>
LS_COLORS=<stuff>
MAIL=/var/mail/<username>
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
PWD=/home/<username>
LANG=de_DE.UTF-8
SHLVL=1
HOME=/home/<username>
LOGNAME=<username>
SSH_CONNECTION=<ip & port>
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #4 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:48:07 UTC ---
Apparently I can't edit my comments, so new comment: With the following changes
to the attached config:

262c262
< BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
---
> BR2_TARGET_GENERIC_GETTY_PORT="tty1"
652,653c652,653
< BR2_PACKAGE_PYTHON3_PY_ONLY=y
< # BR2_PACKAGE_PYTHON3_PYC_ONLY is not set
---
> # BR2_PACKAGE_PYTHON3_PY_ONLY is not set
> BR2_PACKAGE_PYTHON3_PYC_ONLY=y

find output/target/ -name "_ssl*" still is empty (after make clean, make)

output/build/host-python3-3.4.1/build/lib.linux-x86_64-3.4/_ssl.cpython-34m.so
(host-python ssl) gets built though (in every configuration I tried so far).

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-11-14 08:25:37 UTC ---
Thanks for your feedback. Interesting, it clearly doesn't match what I'm seeing
here. Can you post to some pastebin site or some other location the complete
build log? Something like:

  make 2>&1 | tee logfile

And then post 'logfile' somewhere. It would be even nicer if you could after
the build do:

  find output/target/ | tee -a logfile

So that I can also see the full list of files in your target filesystem. I
indeed think something in your host system might be causing the problem, we
need to figure out what.

Thanks again!

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #6 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-11-14 09:06:30 UTC ---
Possibly related to bug #7478

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #7 from tobias.meggendorfer+buildroot at gmail.com 2014-11-14 10:29:36 UTC ---
Again, _ssl* missing in target but present in host. See line 61876, "The
necessary bits to build these optional modules were not found:"

I uploaded this compressed file to my gdrive, the uncompressed log has ~23 mb,
which probably does not get accepted by any pastebin-ish hoster.
https://drive (dot) google (dot)
com/file/d/0BwB7SQYhX6MuZlIwYXdjSDZnZzA/view?usp=sharing

Note: /project is a symlink to /var/data/project on my filesystem (buildroot is
too big for my root partition). I just tried compiling after switching to the
"real" path, just in case, but same result.

I used the config mentioned in comment 4

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #8 from tobias.meggendorfer+buildroot at gmail.com 2014-11-20 17:46:19 UTC ---
Update: I just setup a new virtual machine (3.13.0-32-generic, Ubuntu 14.04 LTS
Server), and apparently there the SSL module gets built and I can run "import
ssl" without a problem.

The used config differs from the one I uploaded but it didn't work with this
particular one on my original system, too.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7628] Python SSL does not get built for Raspberry Pi

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7628

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #9 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2015-06-03 01:31:47 UTC ---
I believe it might have been a locale related issue, since having a non-english
locale would make the Python add_gcc_paths() logic to fail, and therefore it
wouldn't find the headers for OpenSSL.

This has been solved by commit
http://git.buildroot.net/buildroot/commit/package/python?id=238bf198cf4f31a3301b81a924d5e22044945955.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.