[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Sébastien Royen]

setup.py: do not add crosscompile header locations if language is not
english

With buildroot toolchain, gcc can be multi language.
Python package setup.py need gcc english output to work fine. (find zlib
for example)
We force language to en_US for the need of output parsing.

Signed-off-by: Sebastien Royen <sebastien.royen@armadeus.com>
Index: b/setup.py
===================================================================
--- a/setup.py    2015-03-27 23:55:53.738987211 +0100
+++ b/setup.py    2015-03-27 23:44:48.482968892 +0100
@@ -414,7 +414,7 @@
         tmpfile = os.path.join(self.build_temp, 'gccpaths')
         if not os.path.exists(self.build_temp):
             os.makedirs(self.build_temp)
-        ret = os.system('%s -E -v - </dev/null 2>%s 1>/dev/null' %
(gcc, tmpfile))
+        ret = os.system('LANGUAGE=en_US %s -E -v - </dev/null 2>%s
1>/dev/null' % (gcc, tmpfile))
         is_gcc = False
         in_incdirs = False
         inc_dirs = []

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Thomas Petazzoni]

Dear S?bastien Royen,

On Sat, 28 Mar 2015 00:07:24 +0100, S?bastien Royen wrote:
> setup.py: do not add crosscompile header locations if language is not
> english
> 
> With buildroot toolchain, gcc can be multi language.
> Python package setup.py need gcc english output to work fine. (find zlib
> for example)
> We force language to en_US for the need of output parsing.
> 
> Signed-off-by: Sebastien Royen <sebastien.royen@armadeus.com>

What you posted is not a Buildroot patch, so we cannot apply it. I
guess it's a python or python3 patch. Can you rework this to make it
actually usable by Buildroot, and also make sure whether the issue is
applicable to python, python3, or both?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Sébastien Royen]

Sorry,

I should not submit patch of patch too late in the evening.
I hope the version below is better.
Not tested with python3, and not sure toolchain output language shouln't
be fix in another way to avoid same problem on other packages.

Le 29/03/2015 14:46, Thomas Petazzoni a ?crit :
> Dear S?bastien Royen,
>
> On Sat, 28 Mar 2015 00:07:24 +0100, S?bastien Royen wrote:
>> setup.py: do not add crosscompile header locations if language is not
>> english
>>
>> With buildroot toolchain, gcc can be multi language.
>> Python package setup.py need gcc english output to work fine. (find zlib
>> for example)
>> We force language to en_US for the need of output parsing.
>>
>> Signed-off-by: Sebastien Royen <sebastien.royen@armadeus.com>
> What you posted is not a Buildroot patch, so we cannot apply it. I
> guess it's a python or python3 patch. Can you rework this to make it
> actually usable by Buildroot, and also make sure whether the issue is
> applicable to python, python3, or both?
>
> Thanks,
>
> Thomas
--- a/package/python/116-enforce-cross-compile-headers-search.patch  
 1970-01-01 01:00:00.000000000 +0100
+++ b/package/python/116-enforce-cross-compile-headers-search.patch  
 2015-03-27 23:58:21.478991279 +0100
@@ -0,0 +1,20 @@
+setup.py: do not add crosscompile header locations if language is not
english
+
+With buildroot toolchain, gcc is multi language.
+setup.py need english output to work fine. (find zlib for example)
+We force language to en_US for the need of output parsing.
+
+Signed-off-by: Sebastien Royen <sebastien.royen@armadeus.com>
+Index: b/setup.py
+===================================================================
+--- a/setup.py    2015-03-27 23:55:53.738987211 +0100
++++ b/setup.py    2015-03-27 23:44:48.482968892 +0100
+@@ -414,7 +414,7 @@
+         tmpfile = os.path.join(self.build_temp, 'gccpaths')
+         if not os.path.exists(self.build_temp):
+             os.makedirs(self.build_temp)
+-        ret = os.system('%s -E -v - </dev/null 2>%s 1>/dev/null' %
(gcc, tmpfile))
++        ret = os.system('LANGUAGE=en_US %s -E -v - </dev/null 2>%s
1>/dev/null' % (gcc, tmpfile))
+         is_gcc = False
+         in_incdirs = False
+         inc_dirs = []

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Thomas Petazzoni]

Dear S?bastien Royen,

On Sun, 29 Mar 2015 21:08:16 +0200, S?bastien Royen wrote:

> I should not submit patch of patch too late in the evening.
> I hope the version below is better.

Yes and no. It's better because it's a Buildroot patch, but it's still
not good since it's not a Git formatted patch.

> Not tested with python3, and not sure toolchain output language shouln't
> be fix in another way to avoid same problem on other packages.

Maybe we should export LC_ALL globally, or something like that.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Yegor Yefremov]

On Sun, Mar 29, 2015 at 10:40 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear S?bastien Royen,
>
> On Sun, 29 Mar 2015 21:08:16 +0200, S?bastien Royen wrote:
>
>> I should not submit patch of patch too late in the evening.
>> I hope the version below is better.
>
> Yes and no. It's better because it's a Buildroot patch, but it's still
> not good since it's not a Git formatted patch.
>
>> Not tested with python3, and not sure toolchain output language shouln't
>> be fix in another way to avoid same problem on other packages.
>
> Maybe we should export LC_ALL globally, or something like that.

Can confirm, that I'm having the same issue with my German openSUSE
12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
yet. Python 2.7.9.

The whole stuff got screwed only lately. I've been using python-dpkt,
that requires PYTHON_ZLIB for ages and had no problems on this
machine.

I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is
also locale related.

Yegor

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Thomas Petazzoni]

Dear Yegor Yefremov,

On Mon, 30 Mar 2015 13:05:09 +0200, Yegor Yefremov wrote:

> Can confirm, that I'm having the same issue with my German openSUSE
> 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
> yet. Python 2.7.9.

Ok.

> The whole stuff got screwed only lately. I've been using python-dpkt,
> that requires PYTHON_ZLIB for ages and had no problems on this
> machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02,
2014.11 and see if the issue also happens (or not) ?

> I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is
> also locale related.

Ok.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Yegor Yefremov]

On Mon, Mar 30, 2015 at 1:49 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Yegor Yefremov,
>
> On Mon, 30 Mar 2015 13:05:09 +0200, Yegor Yefremov wrote:
>
>> Can confirm, that I'm having the same issue with my German openSUSE
>> 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
>> yet. Python 2.7.9.
>
> Ok.
>
>> The whole stuff got screwed only lately. I've been using python-dpkt,
>> that requires PYTHON_ZLIB for ages and had no problems on this
>> machine.
>
> Seems weird. Do we have an idea what broke things ? Can you try 2015.02,
> 2014.11 and see if the issue also happens (or not) ?

Hm. Strange. 2014.11 seem to have the same issue. Can it be the change
from Python 2.7.8 to 2.7.9?

# cat /etc/os-release
NAME=Buildroot
VERSION=2014.11
ID=buildroot
VERSION_ID=2014.11
PRETTY_NAME="Buildroot 2014.11"

I'm also working with en englich Xubuntu VM. I can imagine, that I
haven't made updates/cleaning on my suse machine. Most upstream work
was made with Xubuntu.

>> I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is
>> also locale related.
>
> Ok.
>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux, Kernel and Android engineering
> http://free-electrons.com

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Yegor Yefremov]

On Mon, Mar 30, 2015 at 3:37 PM, Yegor Yefremov
<yegorslists@googlemail.com> wrote:
> On Mon, Mar 30, 2015 at 1:49 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Dear Yegor Yefremov,
>>
>> On Mon, 30 Mar 2015 13:05:09 +0200, Yegor Yefremov wrote:
>>
>>> Can confirm, that I'm having the same issue with my German openSUSE
>>> 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
>>> yet. Python 2.7.9.
>>
>> Ok.
>>
>>> The whole stuff got screwed only lately. I've been using python-dpkt,
>>> that requires PYTHON_ZLIB for ages and had no problems on this
>>> machine.
>>
>> Seems weird. Do we have an idea what broke things ? Can you try 2015.02,
>> 2014.11 and see if the issue also happens (or not) ?
>
> Hm. Strange. 2014.11 seem to have the same issue. Can it be the change
> from Python 2.7.8 to 2.7.9?
>
> # cat /etc/os-release
> NAME=Buildroot
> VERSION=2014.11
> ID=buildroot
> VERSION_ID=2014.11
> PRETTY_NAME="Buildroot 2014.11"
>
> I'm also working with en englich Xubuntu VM. I can imagine, that I
> haven't made updates/cleaning on my suse machine. Most upstream work
> was made with Xubuntu.

Python 3.4.2 seems to have this issue too.

Yegor

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Yegor Yefremov]

On Mon, Mar 30, 2015 at 4:05 PM, Yegor Yefremov
<yegorslists@googlemail.com> wrote:
> On Mon, Mar 30, 2015 at 3:37 PM, Yegor Yefremov
> <yegorslists@googlemail.com> wrote:
>> On Mon, Mar 30, 2015 at 1:49 PM, Thomas Petazzoni
>> <thomas.petazzoni@free-electrons.com> wrote:
>>> Dear Yegor Yefremov,
>>>
>>> On Mon, 30 Mar 2015 13:05:09 +0200, Yegor Yefremov wrote:
>>>
>>>> Can confirm, that I'm having the same issue with my German openSUSE
>>>> 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
>>>> yet. Python 2.7.9.
>>>
>>> Ok.
>>>
>>>> The whole stuff got screwed only lately. I've been using python-dpkt,
>>>> that requires PYTHON_ZLIB for ages and had no problems on this
>>>> machine.
>>>
>>> Seems weird. Do we have an idea what broke things ? Can you try 2015.02,
>>> 2014.11 and see if the issue also happens (or not) ?
>>
>> Hm. Strange. 2014.11 seem to have the same issue. Can it be the change
>> from Python 2.7.8 to 2.7.9?
>>
>> # cat /etc/os-release
>> NAME=Buildroot
>> VERSION=2014.11
>> ID=buildroot
>> VERSION_ID=2014.11
>> PRETTY_NAME="Buildroot 2014.11"
>>
>> I'm also working with en englich Xubuntu VM. I can imagine, that I
>> haven't made updates/cleaning on my suse machine. Most upstream work
>> was made with Xubuntu.
>
> Python 3.4.2 seems to have this issue too.

Tried to build image via Sourcery CodeBench ARM 2014.05. Python 2.7.9
shows no problems. But when I'm doing the same with BR's toolchain,
zlib module won't be found at runtime.

BR2_arm=y
BR2_cortex_a8=y
BR2_CCACHE=y
BR2_TOOLCHAIN_BUILDROOT_LARGEFILE=y
BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
BR2_TOOLCHAIN_BUILDROOT_INET_RPC=y
BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_BINUTILS_VERSION_2_25=y
BR2_GCC_VERSION_4_9_X=y
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ZLIB=y

[Buildroot] [PATCH] python modules search in non english locale (zlib, ...)

[Sébastien Royen]

Le 31/03/2015 11:05, Yegor Yefremov a ?crit :
> On Mon, Mar 30, 2015 at 4:05 PM, Yegor Yefremov
> <yegorslists@googlemail.com> wrote:
>> On Mon, Mar 30, 2015 at 3:37 PM, Yegor Yefremov
>> <yegorslists@googlemail.com> wrote:
>>> On Mon, Mar 30, 2015 at 1:49 PM, Thomas Petazzoni
>>> <thomas.petazzoni@free-electrons.com> wrote:
>>>> Dear Yegor Yefremov,
>>>>
>>>> On Mon, 30 Mar 2015 13:05:09 +0200, Yegor Yefremov wrote:
>>>>
>>>>> Can confirm, that I'm having the same issue with my German openSUSE
>>>>> 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL
>>>>> yet. Python 2.7.9.
>>>> Ok.
>>>>
>>>>> The whole stuff got screwed only lately. I've been using python-dpkt,
>>>>> that requires PYTHON_ZLIB for ages and had no problems on this
>>>>> machine.
>>>> Seems weird. Do we have an idea what broke things ? Can you try 2015.02,
>>>> 2014.11 and see if the issue also happens (or not) ?
>>> Hm. Strange. 2014.11 seem to have the same issue. Can it be the change
>>> from Python 2.7.8 to 2.7.9?
>>>
>>> # cat /etc/os-release
>>> NAME=Buildroot
>>> VERSION=2014.11
>>> ID=buildroot
>>> VERSION_ID=2014.11
>>> PRETTY_NAME="Buildroot 2014.11"
>>>
>>> I'm also working with en englich Xubuntu VM. I can imagine, that I
>>> haven't made updates/cleaning on my suse machine. Most upstream work
>>> was made with Xubuntu.
>> Python 3.4.2 seems to have this issue too.
> Tried to build image via Sourcery CodeBench ARM 2014.05. Python 2.7.9
> shows no problems. But when I'm doing the same with BR's toolchain,
> zlib module won't be found at runtime.
>
> BR2_arm=y
> BR2_cortex_a8=y
> BR2_CCACHE=y
> BR2_TOOLCHAIN_BUILDROOT_LARGEFILE=y
> BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
> BR2_TOOLCHAIN_BUILDROOT_INET_RPC=y
> BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
> BR2_BINUTILS_VERSION_2_25=y
> BR2_GCC_VERSION_4_9_X=y
> BR2_PACKAGE_PYTHON=y
> BR2_PACKAGE_PYTHON_ZLIB=y
>
Hi,

Problem is on parsing output of : gcc -E -v - </dev/null 1>/dev/null
Maybe it would be better to fix setup.py in order to be generic.
Output with en_US is like :
...
#include <...> search starts here:
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed
 /my_path/bsp/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include
 /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
End of search list.
...

Output with de is like :
...
#include <...> - Suche beginnt hier:
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include
 /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
Ende der Suchliste.
...

Output with fr version is :
la recherche pour #include <...> d?bute ici:
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed
 /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include
 /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
Fin de la liste de recherche.
...

Or maybe we should keep only C language in toolchain build ? ( something
to do with BR2_ENABLE_LOCALE_WHITELIST ? )

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20150331/1efe615f/attachment.html>