Updating daisy -> Python.h not found

Updating daisy -> Python.h not found

[Neuer User]

Hi

Because of shellshock I decided to update my daisy repo to latest
revision (old version was from about June).

Everybody know: "NEVER EVER CHANGE A RUNNING SYSTEM" :-)
Well, I needed to because of shellshock. And now I need to try to repair
everything that broke (currently at 7 packages).

One that I am completely stuck with is a custom recipe (which of course
worked nicely before the repo sync). This is the recipe:

pyotherside.bb:
---------------
SUMMARY = "Python3 bindings for Qt5"
DESCRIPTION = "New framework for including python interpreter into Qt5 apps"
HOMEPAGE = "http://thp.io/2011/pyotherside/"
LICENSE = "MIT"
LIC_FILES_CHKSUM =
"file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

PV = "1.3+git${SRCPV}"

SRCREV = "f870396346bec40b1bb64f95e27d4e3dafeda799"

SRC_URI = "git://github.com/thp/pyotherside.git;branch=master;protocol=git"

S = "${WORKDIR}/git/"

DEPENDS = "qtdeclarative python3 python3-native"
RDEPENDS_${PN} = "qtdeclarative-qmlplugins"

inherit python3native

require recipes-qt/qt5/qt5.inc


I did need to bbappend the original python3 recipe with this here (just
the last part is changed), because python-config used python2.7 instead
of 3.3:

python3-native_3.3.3.bbappend:
------------------------------
do_install() {
        install -d ${D}${libdir}/pkgconfig
        oe_runmake 'DESTDIR=${D}' install
        if [ -e ${WORKDIR}/sitecustomize.py ]; then
                install -m 0644 ${WORKDIR}/sitecustomize.py
${D}/${libdir}/python${PYTHON_MAJMIN}
        fi
        install -d ${D}${bindir}/${PN}
        install -m 0755 Parser/pgen ${D}${bindir}/${PN}

        # Make sure we use /usr/bin/env python3
        for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python
${D}${bindir}/${PN}`; do
                sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
        done
}


Unfortunately, after upgrading daisy I get the following error during
compilation:

In file included from ../../git/src/pyotherside_plugin.cpp:19:0:
../../git/src/qpython_priv.h:22:20: fatal error: Python.h: No such file
or directory
 #include "Python.h"
                    ^
compilation terminated.
In file included from ../../git/src/qml_python_bridge.h:22:0,
                 from ../../git/src/qpython.cpp:19:
../../git/src/pyobject_converter.h:24:20: fatal error: Python.h: No such
file or directory
 #include "Python.h"
                    ^
compilation terminated.
make[1]: *** [pyotherside_plugin.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [qpython.o] Error 1
In file included from ../../git/src/qpython_imageprovider.cpp:19:0:
../../git/src/qpython_priv.h:22:20: fatal error: Python.h: No such file
or directory
 #include "Python.h"
                    ^
compilation terminated.
In file included from ../../git/src/qml_python_bridge.h:22:0,
                 from ../../git/src/qpython_priv.cpp:19:
../../git/src/pyobject_converter.h:24:20: fatal error: Python.h: No such
file or directory
 #include "Python.h"
                    ^
compilation terminated.
make[1]: *** [qpython_priv.o] Error 1
make[1]: *** [qpython_imageprovider.o] Error 1
../../git/src/global_libpython_loader.cpp:25:0: warning: "_GNU_SOURCE"
redefined [enabled by default]
 #define _GNU_SOURCE
 ^
<command-line>:0:0: note: this is the location of the previous definition
../../git/src/global_libpython_loader.cpp:31:12: warning: unused
parameter 'size' [-Wunused-parameter]
 static int load_python_globally_callback(struct dl_phdr_info *info,
size_t size, void *data)
            ^
make[1]: Leaving directory
`/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.3+gitAUTOINC+f870396346-r0/build/src'
make: *** [sub-src-make_first] Error 2
ERROR: oe_runmake failed
WARNING: exit code 1 from a shell command.


Does anybody have any idea, why Python.h is no longer found?

$ find tmp/work -name "Python.h"
tmp/work/i686-linux/python3-native/3.3.3-r0.0/sysroot-destdir/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/include/python3.3m/Python.h
tmp/work/i686-linux/python3-native/3.3.3-r0.0/Python-3.3.3/Include/Python.h
tmp/work/i686-linux/python-native/2.7.3-r0.1/Python-2.7.3/Include/Python.h
tmp/work/i686-linux/python-native/2.7.3-r0.1/image/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
tmp/work/i686-linux/python-native/2.7.3-r0.1/sysroot-destdir/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/package/usr/include/python3.3m/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/sysroot-destdir/usr/include/python3.3m/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/packages-split/python3-dev/usr/include/python3.3m/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/Python-2.7.3/Include/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/package/usr/include/python2.7/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/package/usr/src/debug/python/2.7.3-r0.3/Python-2.7.3/Include/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/image/usr/include/python2.7/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/sysroot-destdir/usr/include/python2.7/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/packages-split/python-dbg/usr/src/debug/python/2.7.3-r0.3/Python-2.7.3/Include/Python.h
tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/packages-split/python-dev/usr/include/python2.7/Python.h

Re: Updating daisy -> Python.h not found

[Paul Eggleton]

Hi there,

On Friday 17 October 2014 12:00:23 Neuer User wrote:
> Because of shellshock I decided to update my daisy repo to latest
> revision (old version was from about June).
> 
> Everybody know: "NEVER EVER CHANGE A RUNNING SYSTEM" :-)
> Well, I needed to because of shellshock. And now I need to try to repair
> everything that broke (currently at 7 packages).
> 
> One that I am completely stuck with is a custom recipe (which of course
> worked nicely before the repo sync). This is the recipe:
> 
> pyotherside.bb:
> ---------------
> SUMMARY = "Python3 bindings for Qt5"
> DESCRIPTION = "New framework for including python interpreter into Qt5 apps"
> HOMEPAGE = "http://thp.io/2011/pyotherside/"
> LICENSE = "MIT"
> LIC_FILES_CHKSUM =
> "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
> 
> PV = "1.3+git${SRCPV}"
> 
> SRCREV = "f870396346bec40b1bb64f95e27d4e3dafeda799"
> 
> SRC_URI = "git://github.com/thp/pyotherside.git;branch=master;protocol=git"
> 
> S = "${WORKDIR}/git/"
> 
> DEPENDS = "qtdeclarative python3 python3-native"
> RDEPENDS_${PN} = "qtdeclarative-qmlplugins"
> 
> inherit python3native
> 
> require recipes-qt/qt5/qt5.inc
> 
> 
> I did need to bbappend the original python3 recipe with this here (just
> the last part is changed), because python-config used python2.7 instead
> of 3.3:
> 
> python3-native_3.3.3.bbappend:
> ------------------------------
> do_install() {
>         install -d ${D}${libdir}/pkgconfig
>         oe_runmake 'DESTDIR=${D}' install
>         if [ -e ${WORKDIR}/sitecustomize.py ]; then
>                 install -m 0644 ${WORKDIR}/sitecustomize.py
> ${D}/${libdir}/python${PYTHON_MAJMIN}
>         fi
>         install -d ${D}${bindir}/${PN}
>         install -m 0755 Parser/pgen ${D}${bindir}/${PN}
> 
>         # Make sure we use /usr/bin/env python3
>         for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python
> ${D}${bindir}/${PN}`; do
>                 sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>         done
> }
> 
> 
> Unfortunately, after upgrading daisy I get the following error during
> compilation:
> 
> In file included from ../../git/src/pyotherside_plugin.cpp:19:0:
> ../../git/src/qpython_priv.h:22:20: fatal error: Python.h: No such file
> or directory
>  #include "Python.h"
>                     ^
> compilation terminated.
> In file included from ../../git/src/qml_python_bridge.h:22:0,
>                  from ../../git/src/qpython.cpp:19:
> ../../git/src/pyobject_converter.h:24:20: fatal error: Python.h: No such
> file or directory
>  #include "Python.h"
>                     ^
> compilation terminated.
> make[1]: *** [pyotherside_plugin.o] Error 1
> make[1]: *** Waiting for unfinished jobs....
> make[1]: *** [qpython.o] Error 1
> In file included from ../../git/src/qpython_imageprovider.cpp:19:0:
> ../../git/src/qpython_priv.h:22:20: fatal error: Python.h: No such file
> or directory
>  #include "Python.h"
>                     ^
> compilation terminated.
> In file included from ../../git/src/qml_python_bridge.h:22:0,
>                  from ../../git/src/qpython_priv.cpp:19:
> ../../git/src/pyobject_converter.h:24:20: fatal error: Python.h: No such
> file or directory
>  #include "Python.h"
>                     ^
> compilation terminated.
> make[1]: *** [qpython_priv.o] Error 1
> make[1]: *** [qpython_imageprovider.o] Error 1
> ../../git/src/global_libpython_loader.cpp:25:0: warning: "_GNU_SOURCE"
> redefined [enabled by default]
>  #define _GNU_SOURCE
>  ^
> <command-line>:0:0: note: this is the location of the previous definition
> ../../git/src/global_libpython_loader.cpp:31:12: warning: unused
> parameter 'size' [-Wunused-parameter]
>  static int load_python_globally_callback(struct dl_phdr_info *info,
> size_t size, void *data)
>             ^
> make[1]: Leaving directory
> `/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/py
> otherside/1.3+gitAUTOINC+f870396346-r0/build/src' make: ***
> [sub-src-make_first] Error 2
> ERROR: oe_runmake failed
> WARNING: exit code 1 from a shell command.
> 
> 
> Does anybody have any idea, why Python.h is no longer found?
> 
> $ find tmp/work -name "Python.h"
> tmp/work/i686-linux/python3-native/3.3.3-r0.0/sysroot-destdir/home/ubuntu/yo
> cto/build/tmp/sysroots/i686-linux/usr/include/python3.3m/Python.h
> tmp/work/i686-linux/python3-native/3.3.3-r0.0/Python-3.3.3/Include/Python.h
> tmp/work/i686-linux/python-native/2.7.3-r0.1/Python-2.7.3/Include/Python.h
> tmp/work/i686-linux/python-native/2.7.3-r0.1/image/home/ubuntu/yocto/build/
> tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
> tmp/work/i686-linux/python-native/2.7.3-r0.1/sysroot-destdir/home/ubuntu/yo
> cto/build/tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/package/
> usr/include/python3.3m/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/sysroot-
> destdir/usr/include/python3.3m/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/packages
> -split/python3-dev/usr/include/python3.3m/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/Python-2.
> 7.3/Include/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/package/u
> sr/include/python2.7/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/package/u
> sr/src/debug/python/2.7.3-r0.3/Python-2.7.3/Include/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/image/usr
> /include/python2.7/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/sysroot-d
> estdir/usr/include/python2.7/Python.h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/packages-
> split/python-dbg/usr/src/debug/python/2.7.3-r0.3/Python-2.7.3/Include/Python
> .h
> tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python/2.7.3-r0.3/packages-
> split/python-dev/usr/include/python2.7/Python.h

I agree this sort of thing shouldn't happen on a stable branch. I'm not sure 
what would have caused such a problem though. However, above you are looking 
under tmp/work, but the file being present under there doesn't really say much 
- the key question is is it still in the sysroot i.e. tmp/sysroots/* ? The 
first thing to figure out is if the file is really missing, or if it is now 
simply not looking for it in the right location.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Updating daisy -> Python.h not found

[Neuer User]

Am 17.10.2014 um 16:51 schrieb Paul Eggleton:
> Hi there,
> 
> I agree this sort of thing shouldn't happen on a stable branch. I'm not sure 
> what would have caused such a problem though. However, above you are looking 
> under tmp/work, but the file being present under there doesn't really say much 
> - the key question is is it still in the sysroot i.e. tmp/sysroots/* ? The 
> first thing to figure out is if the file is really missing, or if it is now 
> simply not looking for it in the right location.
> 
> Cheers,
> Paul
> 
Very good point.

$ find tmp/sysroots/ -name "Python.h"
tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
tmp/sysroots/i686-linux/usr/include/python3.3m/Python.h
tmp/sysroots/cubox-i/usr/include/python2.7/Python.h
tmp/sysroots/cubox-i/usr/include/python3.3m/Python.h

So, it seems to be there, if I see that correctly. The question then
seems to be, why it is no longer found?

Thanks for you help, Paul.

Cheers

Michael

Re: Updating daisy -> Python.h not found

[Paul Eggleton]

On Friday 17 October 2014 17:01:46 Neuer User wrote:
> Am 17.10.2014 um 16:51 schrieb Paul Eggleton:
> > Hi there,
> > 
> > I agree this sort of thing shouldn't happen on a stable branch. I'm not
> > sure what would have caused such a problem though. However, above you are
> > looking under tmp/work, but the file being present under there doesn't
> > really say much - the key question is is it still in the sysroot i.e.
> > tmp/sysroots/* ? The first thing to figure out is if the file is really
> > missing, or if it is now simply not looking for it in the right location.
> > 
> > Cheers,
> > Paul
> 
> Very good point.
> 
> $ find tmp/sysroots/ -name "Python.h"
> tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
> tmp/sysroots/i686-linux/usr/include/python3.3m/Python.h
> tmp/sysroots/cubox-i/usr/include/python2.7/Python.h
> tmp/sysroots/cubox-i/usr/include/python3.3m/Python.h
> 
> So, it seems to be there, if I see that correctly. The question then
> seems to be, why it is no longer found?

I'm not sure. That's where you'd need to look at how the software your recipe 
is building searches for that - the configure log, devshell might be useful for 
determining that.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Updating daisy -> Python.h not found

[Neuer User]

Hmm, I looked through the compile log before, but you are right: the
configure log might be more interesting:

There is one error during configure (which, however, did not abort the
process, strangely):

DEBUG: Executing python function sysroot_cleansstate
DEBUG: Python function sysroot_cleansstate finished
DEBUG: Executing shell function qmake5_base_preconfigure
DEBUG: Shell function qmake5_base_preconfigure finished
DEBUG: Executing shell function do_configure
Reading
/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.3+gitAUTOINC+f870396346-r0/git/src/src.pro
[/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.3+gitAUTOINC+f870396346-r0/build/src]
Project MESSAGE: PYTHON_CONFIG = python3-config
Traceback (most recent call last):
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/bin/python3-native/python3-config",
line 7, in <module>
    from distutils import sysconfig
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/lib/python3.3/distutils/sysconfig.py",
line 19, in <module>
    PREFIX = os.path.normpath(sys.prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: Can't convert 'NoneType' object to str implicitly
Traceback (most recent call last):
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/bin/python3-native/python3-config",
line 7, in <module>
    from distutils import sysconfig
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/lib/python3.3/distutils/sysconfig.py",
line 19, in <module>
    PREFIX = os.path.normpath(sys.prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: Can't convert 'NoneType' object to str implicitly
Reading
/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.3+gitAUTOINC+f870396346-r0/git/tests/tests.pro
[/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.3+gitAUTOINC+f870396346-r0/build/tests]
Project MESSAGE: PYTHON_CONFIG = python3-config
Traceback (most recent call last):
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/bin/python3-native/python3-config",
line 7, in <module>
    from distutils import sysconfig
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/lib/python3.3/distutils/sysconfig.py",
line 19, in <module>
    PREFIX = os.path.normpath(sys.prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: Can't convert 'NoneType' object to str implicitly
Traceback (most recent call last):
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/bin/python3-native/python3-config",
line 7, in <module>
    from distutils import sysconfig
  File
"/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/lib/python3.3/distutils/sysconfig.py",
line 19, in <module>
    PREFIX = os.path.normpath(sys.prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: Can't convert 'NoneType' object to str implicitly
DEBUG: Shell function do_configure finished
DEBUG: Executing python function do_qa_configure
NOTE: Checking autotools environment for common misconfiguration
NOTE: pyotherside: md5 checksum matched for
file:///home/ubuntu/yocto/sources/poky/meta/files/common-licenses/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302
DEBUG: Python function do_qa_configure finished
DEBUG: Executing shell function qmake5_base_postconfigure
DEBUG: Shell function qmake5_base_postconfigure finished

Guess this not what it should be, right?



Am 17.10.2014 um 17:05 schrieb Paul Eggleton:
> On Friday 17 October 2014 17:01:46 Neuer User wrote:
>> Am 17.10.2014 um 16:51 schrieb Paul Eggleton:
>>> Hi there,
>>>
>>> I agree this sort of thing shouldn't happen on a stable branch. I'm not
>>> sure what would have caused such a problem though. However, above you are
>>> looking under tmp/work, but the file being present under there doesn't
>>> really say much - the key question is is it still in the sysroot i.e.
>>> tmp/sysroots/* ? The first thing to figure out is if the file is really
>>> missing, or if it is now simply not looking for it in the right location.
>>>
>>> Cheers,
>>> Paul
>>
>> Very good point.
>>
>> $ find tmp/sysroots/ -name "Python.h"
>> tmp/sysroots/i686-linux/usr/include/python2.7/Python.h
>> tmp/sysroots/i686-linux/usr/include/python3.3m/Python.h
>> tmp/sysroots/cubox-i/usr/include/python2.7/Python.h
>> tmp/sysroots/cubox-i/usr/include/python3.3m/Python.h
>>
>> So, it seems to be there, if I see that correctly. The question then
>> seems to be, why it is no longer found?
> 
> I'm not sure. That's where you'd need to look at how the software your recipe 
> is building searches for that - the configure log, devshell might be useful for 
> determining that.
> 
> Cheers,
> Paul
> 

Re: Updating daisy -> Python.h not found

[Neuer User]

Hmm, going deeper into the configure error I found the following:

File
/home/ubuntu/yocto/build/tmp/sysroots/i686-linux/usr/lib/python3.3/distutils/sysconfig.py
contains in line 19:

PREFIX = os.path.normpath(sys.prefix).replace( os.getenv("BUILD_SYS"),
os.getenv("HOST_SYS") )

This gives error

TypeError: Can't convert 'NoneType' object to str implicitly

It does the same if I type this line on python3 on my host computer.

Looking into the sysconfig of python3 of my ubuntu installation, it does
NOT HAVE this line!

here are the lines from python3 sysconfig.py from yocto:

---------------------------------------------------------------------
"""Provide access to Python's configuration information.  The specific
configuration variables available depend heavily on the platform and
configuration.  The values may be retrieved using
get_config_var(name), and the list of variables is available via
get_config_vars().keys().  Additional convenience functions are also
available.

Written by:   Fred L. Drake, Jr.
Email:        <fdrake@acm.org>
"""

import os
import re
import sys

from .errors import DistutilsPlatformError

# These are needed in a couple of spots, so just compute them once.
PREFIX = os.path.normpath(sys.prefix).replace( os.getenv("BUILD_SYS"),
os.getenv("HOST_SYS") )
EXEC_PREFIX = os.path.normpath(sys.exec_prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
BASE_PREFIX = os.path.normpath(sys.base_prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
BASE_EXEC_PREFIX= os.path.normpath(sys.base_exec_prefix).replace(
os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )

[...]
----------------------------------------------------------------------

and here from my host pc:

----------------------------------------------------------------------
"""Provide access to Python's configuration information.  The specific
configuration variables available depend heavily on the platform and
configuration.  The values may be retrieved using
get_config_var(name), and the list of variables is available via
get_config_vars().keys().  Additional convenience functions are also
available.

Written by:   Fred L. Drake, Jr.
Email:        <fdrake@acm.org>
"""

import os
import re
import sys

from .errors import DistutilsPlatformError

# These are needed in a couple of spots, so just compute them once.
PREFIX = os.path.normpath(sys.prefix)
EXEC_PREFIX = os.path.normpath(sys.exec_prefix)
BASE_PREFIX = os.path.normpath(sys.base_prefix)
BASE_EXEC_PREFIX = os.path.normpath(sys.base_exec_prefix)
----------------------------------------------------------------------


Now, I don't know if that changed with the daisy update, but it
definitely seems to be wrong.

Any python3 expert out there? Can we find out (git blame) when these
lines were changed last?


EDIT: found it, it's this patch:

./poky/meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch

and it is from 2013-05-14, so pretty old... :-(

That leaves the question why it gives an error now after updating daisy,
but not before.

Here is the log_configure from before updating daisy:

DEBUG: Executing python function sysroot_cleansstate
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/sysroot-providers/pyotherside
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/io/thp/pyotherside/qmldir
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/io/thp/pyotherside/libpyothersideplugin.so
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/io/thp/pyotherside/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/io/thp/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/io/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/sysroot-providers/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/qml/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/qt5/
DEBUG: Removing manifest:
/home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/lib/
DEBUG: Removing manifest: /home/ubuntu/yocto/build/tmp/sysroots/cubox-i/usr/
DEBUG: Python function sysroot_cleansstate finished
DEBUG: Executing shell function qmake5_base_preconfigure
Previously configured separate build directory detected, cleaning
/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.2+gitAUTOINC+87e5eaf470-r0/build
DEBUG: Shell function qmake5_base_preconfigure finished
DEBUG: Executing shell function do_configure
Reading
/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.2+gitAUTOINC+87e5eaf470-r0/git/src/src.pro
[/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.2+gitAUTOINC+87e5eaf470-r0/build/src]
Project MESSAGE: PYTHON_CONFIG = python3-config
Reading
/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.2+gitAUTOINC+87e5eaf470-r0/git/tests/tests.pro
[/home/ubuntu/yocto/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/pyotherside/1.2+gitAUTOINC+87e5eaf470-r0/build/tests]
Project MESSAGE: PYTHON_CONFIG = python3-config
DEBUG: Shell function do_configure finished
DEBUG: Executing python function do_qa_configure
NOTE: Checking autotools environment for common misconfiguration
NOTE: pyotherside: md5 checksum matched for
file:///home/ubuntu/yocto/sources/poky/meta/files/common-licenses/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302
DEBUG: Python function do_qa_configure finished
DEBUG: Executing shell function qmake5_base_postconfigure
DEBUG: Shell function qmake5_base_postconfigure finished

NO ERROR...

Hmmm...

Re: Updating daisy -> Python.h not found

[Neuer User]

I don't see any chance to get this running. The whole environment seems
to be missing, such as BUILD_SYS, HOST_SYS, STAGING_LIB_DIR and other.

I will now completely resetup yocto from scratch, hoping that this fixes
the problem. Will report tomorrow or as soon as I have a result.

Re: Updating daisy -> Python.h not found

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 680 bytes --]

On Sat, Oct 18, 2014 at 09:11:51AM +0200, Neuer User wrote:
> I don't see any chance to get this running. The whole environment seems
> to be missing, such as BUILD_SYS, HOST_SYS, STAGING_LIB_DIR and other.
> 
> I will now completely resetup yocto from scratch, hoping that this fixes
> the problem. Will report tomorrow or as soon as I have a result.

Have you tried to revert these 2 changes?
http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861
http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=ae49adc13db10cb39eeb9377eb4c60a4db436e00

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Updating daisy -> Python.h not found

[Neuer User]

Am 18.10.2014 um 10:44 schrieb Martin Jansa:
> On Sat, Oct 18, 2014 at 09:11:51AM +0200, Neuer User wrote:
>> I don't see any chance to get this running. The whole environment seems
>> to be missing, such as BUILD_SYS, HOST_SYS, STAGING_LIB_DIR and other.
>>
>> I will now completely resetup yocto from scratch, hoping that this fixes
>> the problem. Will report tomorrow or as soon as I have a result.
> 
> Have you tried to revert these 2 changes?
> http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861
> http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=ae49adc13db10cb39eeb9377eb4c60a4db436e00
> 
> 
> 
Thanks, they look very promising.

Currently I have completely set up my yocto environment from scratch
again. It's curently compiling. There was another error that due to some
metadata left from the older daisy version that I needed to fix by
cleaning the work dir. So maybe this is something similar here.

If that doesn't help, I will look into reverting these two changes.


Btw. do you know that the qt5 repo is currently incompatible with daisy?
There is a patch in fsl-arm that cannot be applied:

./meta-fsl-arm/qt5-layer/recipes-qt/qt5/qtmultimedia_%.bbappend

This needs to be deleted, otherwise qtmultimedia cannot be generated.

Re: Updating daisy -> Python.h not found

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]

On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
> Am 18.10.2014 um 10:44 schrieb Martin Jansa:
> > On Sat, Oct 18, 2014 at 09:11:51AM +0200, Neuer User wrote:
> >> I don't see any chance to get this running. The whole environment seems
> >> to be missing, such as BUILD_SYS, HOST_SYS, STAGING_LIB_DIR and other.
> >>
> >> I will now completely resetup yocto from scratch, hoping that this fixes
> >> the problem. Will report tomorrow or as soon as I have a result.
> > 
> > Have you tried to revert these 2 changes?
> > http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861
> > http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=ae49adc13db10cb39eeb9377eb4c60a4db436e00
> > 
> > 
> > 
> Thanks, they look very promising.
> 
> Currently I have completely set up my yocto environment from scratch
> again. It's curently compiling. There was another error that due to some
> metadata left from the older daisy version that I needed to fix by
> cleaning the work dir. So maybe this is something similar here.
> 
> If that doesn't help, I will look into reverting these two changes.
> 
> 
> Btw. do you know that the qt5 repo is currently incompatible with daisy?
> There is a patch in fsl-arm that cannot be applied:
> 
> ./meta-fsl-arm/qt5-layer/recipes-qt/qt5/qtmultimedia_%.bbappend
> 
> This needs to be deleted, otherwise qtmultimedia cannot be generated.

Are you using meta-qt5/daisy or meta-qt5/master?

Please report it to meta-fsl-arm maintainer, I'm not using that layer,
but I think they should either provide separate .bbappends for _5.*.bb and
*_git.bb or make sure it applies for both versions (there is 5.3.2 and
5.4-alpha in meta-qt5/master and many patches need to be slightly
different).

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Updating daisy -> Python.h not found

[Neuer User]

Am 18.10.2014 um 11:10 schrieb Martin Jansa:
> On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
>> Btw. do you know that the qt5 repo is currently incompatible with daisy?
>> There is a patch in fsl-arm that cannot be applied:
>>
>> ./meta-fsl-arm/qt5-layer/recipes-qt/qt5/qtmultimedia_%.bbappend
>>
>> This needs to be deleted, otherwise qtmultimedia cannot be generated.
> 
> Are you using meta-qt5/daisy or meta-qt5/master?
> 
Hmm, master. Didn't know there was a daisy branch. Did a diff between
the two branches now and do not see any difference though which could
solve the above problem (just seems to add some things such as
noqtwebkit option). I will try it though.

> Please report it to meta-fsl-arm maintainer, I'm not using that layer,
> but I think they should either provide separate .bbappends for _5.*.bb and
> *_git.bb or make sure it applies for both versions (there is 5.3.2 and
> 5.4-alpha in meta-qt5/master and many patches need to be slightly
> different).
> 

Ok. will send that to the maintainer.

Cheers

Michael


P.S.: My yocto install is still compiling but I already saw that the
error is still there. So I will now try to revert the two patches you
mentioned.

Re: Updating daisy -> Python.h not found

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2647 bytes --]

On Sat, Oct 18, 2014 at 11:19:14AM +0200, Neuer User wrote:
> Am 18.10.2014 um 11:10 schrieb Martin Jansa:
> > On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
> >> Btw. do you know that the qt5 repo is currently incompatible with daisy?
> >> There is a patch in fsl-arm that cannot be applied:
> >>
> >> ./meta-fsl-arm/qt5-layer/recipes-qt/qt5/qtmultimedia_%.bbappend
> >>
> >> This needs to be deleted, otherwise qtmultimedia cannot be generated.
> > 
> > Are you using meta-qt5/daisy or meta-qt5/master?
> > 
> Hmm, master. Didn't know there was a daisy branch. Did a diff between
> the two branches now and do not see any difference though which could
> solve the above problem (just seems to add some things such as
> noqtwebkit option). I will try it though.

There is very different version of qtmultimedia and gst-1.0 support

 recipes-qt/qt5/qtmultimedia-git/0001-Initial-porting-effort-to-GStreamer-1.0.patch                  | 2369 ++++++++++++++++++++++++++++++++++++++++++++++++
 recipes-qt/qt5/qtmultimedia-git/0002-qtmultimedia.pro-Respect-OE_GSTREAMER_ENABLED-OE_GST.patch     |   53 ++
 recipes-qt/qt5/qtmultimedia.inc                                                                     |   18 +-
 recipes-qt/qt5/qtmultimedia/0001-Initial-porting-effort-to-GStreamer-1.0.patch                      | 2381 +++++++++++++++++++++++++++++++++++++++++++++++++
 recipes-qt/qt5/qtmultimedia/0002-qtmultimedia.pro-Respect-OE_GSTREAMER_ENABLED-OE_GST.patch         |   53 ++
 recipes-qt/qt5/qtmultimedia_5.2.1.bb                                                                |    5 -
 recipes-qt/qt5/qtmultimedia_5.3.2.bb                                                                |    5 +
 recipes-qt/qt5/qtmultimedia_git.bb                                                                  |   10 +-

> > Please report it to meta-fsl-arm maintainer, I'm not using that layer,
> > but I think they should either provide separate .bbappends for _5.*.bb and
> > *_git.bb or make sure it applies for both versions (there is 5.3.2 and
> > 5.4-alpha in meta-qt5/master and many patches need to be slightly
> > different).
> > 
> 
> Ok. will send that to the maintainer.
> 
> Cheers
> 
> Michael
> 
> 
> P.S.: My yocto install is still compiling but I already saw that the
> error is still there. So I will now try to revert the two patches you
> mentioned.
> 
> 
> -- 
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

SOLUTION: Re: Updating daisy -> Python.h not found

[Neuer User]

Am 18.10.2014 um 11:10 schrieb Martin Jansa:
> On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
>>> http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861

THANKS A LOT!!!

This is it. This patch broke the recipe. I reverted it and it runs
through now without error.

THANKS A LOT!!!

What shall we do now? Is this a bug in this patch? Or does my recipe
need some adaptation?

Re: SOLUTION: Re: Updating daisy -> Python.h not found

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On Sat, Oct 18, 2014 at 11:32:47AM +0200, Neuer User wrote:
> Am 18.10.2014 um 11:10 schrieb Martin Jansa:
> > On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
> >>> http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861
> 
> THANKS A LOT!!!
> 
> This is it. This patch broke the recipe. I reverted it and it runs
> through now without error.
> 
> THANKS A LOT!!!
> 
> What shall we do now? Is this a bug in this patch? Or does my recipe
> need some adaptation?

+ Tyler Hall

My guess is that your app is receiving the include path with sysroot
prefix now from sysconfig and then adds another sysroot prefix so it
points to non-existent directory, check the include paths (e.g. run
make -n in your configured qmake project) and if it's the
case then debug where it's added (OE_QMAKE_DEBUG_OUTPUT = "-d" should
help)

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: SOLUTION: Re: Updating daisy -> Python.h not found

[Tyler Hall]

That patch changed python3-config to use distutils.sysconfig to get
paths. That was already patched by OE to use environment variables to
substitute the correct path, but it seems there's nothing guaranteeing
those variables are exported.

> PREFIX = os.path.normpath(sys.prefix).replace( os.getenv("BUILD_SYS"),
> os.getenv("HOST_SYS") )
>
> This gives error
>
> TypeError: Can't convert 'NoneType' object to str implicitly

It may have worked previously because python3-config returned the path
to the native sysroot and it compiled successfully despite picking up
the header from the wrong place.

There are a few python recipes that export these variables explicitly,
but it doesn't seem like something that clients should have to know
about.

From python-pycurl:
>
> # need to export these variables for python-config to work
> export BUILD_SYS
> export HOST_SYS
> export STAGING_INCDIR
> export STAGING_LIBDIR

This is also done in python-pygobject and python-dbus. Is this
something that distutils.bbclass should do?

-Tyler

Re: SOLUTION: Re: Updating daisy -> Python.h not found

[Tyler Hall]

** Resent to [yocto] as apparently I wasn't subscribed **

That patch changed python3-config to use distutils.sysconfig to get
paths. That was already patched by OE to use environment variables to
substitute the correct path, but it seems there's nothing guaranteeing
those variables are exported.

> PREFIX = os.path.normpath(sys.prefix).
replace( os.getenv("BUILD_SYS"),
> os.getenv("HOST_SYS") )
>
> This gives error
>
> TypeError: Can't convert 'NoneType' object to str implicitly

It may have worked previously because python3-config returned the path
to the native sysroot and it compiled successfully despite picking up
the header from the wrong place.

There are a few python recipes that export these variables explicitly,
but it doesn't seem like something that clients should have to know
about.

From python-pycurl:
>
> # need to export these variables for python-config to work
> export BUILD_SYS
> export HOST_SYS
> export STAGING_INCDIR
> export STAGING_LIBDIR

This is also done in python-pygobject and python-dbus. Is this
something that distutils.bbclass should do?

On Sat, Oct 18, 2014 at 6:03 AM, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Sat, Oct 18, 2014 at 11:32:47AM +0200, Neuer User wrote:
>> Am 18.10.2014 um 11:10 schrieb Martin Jansa:
>> > On Sat, Oct 18, 2014 at 10:51:30AM +0200, Neuer User wrote:
>> >>> http://git.openembedded.org/openembedded-core/commit/?h=daisy&id=de5797b27a358954eb15318d0d77ad1981981861
>>
>> THANKS A LOT!!!
>>
>> This is it. This patch broke the recipe. I reverted it and it runs
>> through now without error.
>>
>> THANKS A LOT!!!
>>
>> What shall we do now? Is this a bug in this patch? Or does my recipe
>> need some adaptation?
>
> + Tyler Hall
>
> My guess is that your app is receiving the include path with sysroot
> prefix now from sysconfig and then adds another sysroot prefix so it
> points to non-existent directory, check the include paths (e.g. run
> make -n in your configured qmake project) and if it's the
> case then debug where it's added (OE_QMAKE_DEBUG_OUTPUT = "-d" should
> help)
>
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: SOLUTION: Re: Updating daisy -> Python.h not found

[Neuer User]

Am 18.10.2014 um 23:33 schrieb Tyler Hall:
>>
>> # need to export these variables for python-config to work
>> export BUILD_SYS
>> export HOST_SYS
>> export STAGING_INCDIR
>> export STAGING_LIBDIR
> 

Tested and this works, when added to my pyotherside recipe.

It is indeed not very intuitive, but writing recipes is anyway something
mysterical (e.g. I need to inherit python3-native, whereas pycurl only
inherits distutils; strange).

Thanks a lot. This is probably the best solution.