[Buildroot] Buildroot 2018.05-rc3 released

[Buildroot] Buildroot 2018.05-rc3 released

[Peter Korsgaard]

Hi,

Buildroot 2018.05-rc3 is released - Go download it at:

http://buildroot.net/downloads/buildroot-2018.05-rc3.tar.gz

or

http://buildroot.net/downloads/buildroot-2018.05-rc3.tar.bz2

Or get it from Git:

git://git.buildroot.net/buildroot

Another week, another release candidate.

Please give it a spin and report any problems to the mailing list or
bug tracker. Many thanks to the people contributing to this release:

git shortlog -sn 2018.05-rc2..
    10  Baruch Siach
     5  Fabio Estevam
     2  Fabrice Fontaine
     2  Peter Korsgaard
     2  Yann E. MORIN
     1  Angelo Compagnucci
     1  Arnout Vandecappelle (Essensium/Mind)
     1  Carlos Santos
     1  Evgeniy Didin
     1  J?r?me Oufella
     1  Matt Weber
     1  Romain Naour
     1  Waldemar Brodkorb

For more details, see the CHANGES file:

http://git.buildroot.net/buildroot/plain/CHANGES?id=2018.05-rc3

Expect the final 2018.05 release by a day or two, unless something
critical shows up.

-- 
Bye, Peter Korsgaard

[Buildroot] Buildroot 2018.05-rc3 released

[Mike Davies]

ca-certificates fails to build. Though I see there is some discussion
on the list, I'm unsure if it is connected to the build failing.
My real build needs a lot more stuff, but it can be trimmed down to
this for it to fail...

tar -xzf /home/mike/Downloads/buildroot-2018.05-rc3.tar.gz
cd b*3
make raspberrypi3_64_defconfig
make menuconfig
Target Packages->Libraries->Crypto->CA Certificates
make
.
.
.

ln -sf python2 /home/mike/work/pi/buildroot-2018.05-rc3/output/host/bin/python
ln -sf python2-config /home/mike/work/pi/buildroot-2018.05-rc3/output/host/bin/python-config
>>> ca-certificates 20170717 Downloading
PATH="/home/mike/work/pi/buildroot-2018.05-rc3/output/host/bin:/home/mike/work/pi/buildroot-2018.05-rc3/output/host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/mike/bin:/sbin:/home/mike/.local/bin:/home/mike/bin:/home/mike/bin:/sbin" BR2_DL_DIR="/home/mike/work/pi/buildroot-2018.05-rc3/dl" BUILD_DIR=/home/mike/work/pi/buildroot-2018.05-rc3/output/build O=/home/mike/work/pi/buildroot-2018.05-rc3/output flock /home/mike/work/pi/buildroot-2018.05-rc3/dl/ca-certificates/ support/download/dl-wrapper -c '20170717' -d '/home/mike/work/pi/buildroot-2018.05-rc3/dl/ca-certificates' -D '/home/mike/work/pi/buildroot-2018.05-rc3/dl' -f 'ca-certificates_20170717.tar.xz' -H 'package/ca-certificates//ca-certificates.hash' -n 'ca-certificates-20170717' -N 'ca-certificates' -o '/home/mike/work/pi/buildroot-2018.05-rc3/dl/ca-certificates/ca-certificates_20170717.tar.xz'  -u http+http://snapshot.debian.org/archive/debian/20170814T210836Z/pool/main/c/ca-certificates -u http\|urlencode+http://sources.buildroot.net/ca-certificates -u http\|urlencode+http://sources.buildroot.net  -- 
--2018-05-29 14:13:13--  http://snapshot.debian.org/archive/debian/20170814T210836Z/pool/main/c/ca-certificates/ca-certificates_20170717.tar.xz
Resolving snapshot.debian.org... 193.62.202.27, 185.17.185.185, 2001:630:206:4000:1a1a:0:c13e:ca1b, ...
Connecting to snapshot.debian.org|193.62.202.27|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 293028 (286K) [application/x-tar]
Saving to: ?/home/mike/work/pi/buildroot-2018.05-rc3/output/build/.ca-certificates_20170717.tar.xz.BjYGHQ/output?

/home/mike/work/pi/buildroot-2018. 100%[=============================================================>] 286.16K  --.-KB/s    in 0.08s   

2018-05-29 14:13:13 (3.31 MB/s) - ?/home/mike/work/pi/buildroot-2018.05-rc3/output/build/.ca-certificates_20170717.tar.xz.BjYGHQ/output? saved [293028/293028]

ca-certificates_20170717.tar.xz: OK (sha1: 4c093ceee1f1428d5aa7e9a5324b92961c0c0f06)
ca-certificates_20170717.tar.xz: OK (sha256: e487639b641fa75445174734dd6e9d600373e3248b3d86a7e3c6d0f6977decd2)
>>> ca-certificates 20170717 Extracting
xzcat /home/mike/work/pi/buildroot-2018.05-rc3/dl/ca-certificates/ca-certificates_20170717.tar.xz | tar --strip-components=1 -C /home/mike/work/pi/buildroot-2018.05-rc3/output/build/ca-certificates-20170717   -xf -
>>> ca-certificates 20170717 Patching
>>> ca-certificates 20170717 Configuring
>>> ca-certificates 20170717 Building
PATH="/home/mike/work/pi/buildroot-2018.05-rc3/output/host/bin:/home/mike/work/pi/buildroot-2018.05-rc3/output/host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/mike/bin:/sbin:/home/mike/.local/bin:/home/mike/bin:/home/mike/bin:/sbin" /usr/bin/make -j9 -C /home/mike/work/pi/buildroot-2018.05-rc3/output/build/ca-certificates-20170717 all
for dir in mozilla; do \
  /usr/bin/make -C $dir all; \
done
python certdata2pem.py
Traceback (most recent call last):
  File "certdata2pem.py", line 23, in <module>
    import base64
  File "/home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/base64.py", line 9, in <module>
    import struct
  File "/home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/struct.py", line 1, in <module>
    from _struct import *
ImportError: /home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/lib-dynload/_struct.so: undefined symbol: PyUnicodeUCS2_AsEncodedString
Makefile:6: recipe for target 'all' failed
make[3]: *** [all] Error 1
Makefile:9: recipe for target 'all' failed
make[2]: *** [all] Error 2
package/pkg-generic.mk:223: recipe for target '/home/mike/work/pi/buildroot-2018.05-rc3/output/build/ca-certificates-20170717/.stamp_built' failed
make[1]: *** [/home/mike/work/pi/buildroot-2018.05-rc3/output/build/ca-certificates-20170717/.stamp_built] Error 2
Makefile:79: recipe for target '_all' failed
make: *** [_all] Error 2


On Tue, 2018-05-29 at 08:44 +0200, Peter Korsgaard wrote:
> Hi,
> 
> Buildroot 2018.05-rc3 is released - Go download it at:
> 
> http://buildroot.net/downloads/buildroot-2018.05-rc3.tar.gz
> 
> or
> 
> http://buildroot.net/downloads/buildroot-2018.05-rc3.tar.bz2
> 
> Or get it from Git:
> 
> git://git.buildroot.net/buildroot
> 
> Another week, another release candidate.
> 
> Please give it a spin and report any problems to the mailing list or
> bug tracker. Many thanks to the people contributing to this release:
> 
> git shortlog -sn 2018.05-rc2..
> ????10??Baruch Siach
> ?????5??Fabio Estevam
> ?????2??Fabrice Fontaine
> ?????2??Peter Korsgaard
> ?????2??Yann E. MORIN
> ?????1??Angelo Compagnucci
> ?????1??Arnout Vandecappelle (Essensium/Mind)
> ?????1??Carlos Santos
> ?????1??Evgeniy Didin
> ?????1??J?r?me Oufella
> ?????1??Matt Weber
> ?????1??Romain Naour
> ?????1??Waldemar Brodkorb
> 
> For more details, see the CHANGES file:
> 
> http://git.buildroot.net/buildroot/plain/CHANGES?id=2018.05-rc3
> 
> Expect the final 2018.05 release by a day or two, unless something
> critical shows up.
> 

[Buildroot] Buildroot 2018.05-rc3 released

[Peter Korsgaard]

>>>>> "Mike" == Mike Davies <busybox_mail@rovoreed.com> writes:

 > ca-certificates fails to build. Though I see there is some discussion
 > on the list, I'm unsure if it is connected to the build failing.
 > My real build needs a lot more stuff, but it can be trimmed down to
 > this for it to fail...

 > tar -xzf /home/mike/Downloads/buildroot-2018.05-rc3.tar.gz
 > cd b*3
 > make raspberrypi3_64_defconfig
 > make menuconfig
 > Target Packages->Libraries->Crypto->CA Certificates
 > make

 > PATH="/home/mike/work/pi/buildroot-2018.05-rc3/output/host/bin:/home/mike/work/pi/buildroot-2018.05-rc3/output/host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/mike/bin:/sbin:/home/mike/.local/bin:/home/mike/bin:/home/mike/bin:/sbin" /usr/bin/make -j9 -C /home/mike/work/pi/buildroot-2018.05-rc3/output/build/ca-certificates-20170717 all
 > for dir in mozilla; do \
 >   /usr/bin/make -C $dir all; \
 > done
 > python certdata2pem.py
 > Traceback (most recent call last):
 >   File "certdata2pem.py", line 23, in <module>
 >     import base64
 >   File "/home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/base64.py", line 9, in <module>
 >     import struct
 >   File "/home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/struct.py", line 1, in <module>
 >     from _struct import *
 > ImportError: /home/mike/work/pi/buildroot-2018.05-rc3/output/host/lib64/python2.7/lib-dynload/_struct.so: undefined symbol: PyUnicodeUCS2_AsEncodedString

Hmm, strange. Nothing has really changed to host-python recently. Is
this a new regression since 2018.05-rc2? What host OS are you using?

-- 
Bye, Peter Korsgaard

[Buildroot] Buildroot 2018.05-rc3 released

[Peter Korsgaard]

>>>>> "Mike" == Mike Davies <busybox_mail@rovoreed.com> writes:

 > Thanks for the reply.
 > Exactly the same error in 2018.05-rc2.

 > I have not tried it anything more recent than buildroot-2017.08 (so
 > that's a long time ago)

 > I am surprised no-one else has picked up on this.

 > If I select Python as a package to install on my target, then it builds
 > OK - but I don't want Python on my target.

 > For now, I guess a workaround is to select Python, and delete it in a
 > pre-image script.

That is odd. The presence or not of python for the target shouldn't have
any effect on host-python. You do not mean python3 for the target, do
you?


 >> ?> ca-certificates fails to build. Though I see there is some
 >> discussion
 >> ?> on the list, I'm unsure if it is connected to the build failing.
 >> ?> My real build needs a lot more stuff, but it can be trimmed down
 >> to
 >> ?> this for it to fail...
 >> 
 >> ?> tar -xzf /home/mike/Downloads/buildroot-2018.05-rc3.tar.gz
 >> ?> cd b*3
 >> ?> make raspberrypi3_64_defconfig
 >> ?> make menuconfig
 >> ?> Target Packages->Libraries->Crypto->CA Certificates
 >> ?> make

I tried it here and it builds without issues:

..
>>>   Executing post-image script board/raspberrypi3-64/post-image.sh
vfat(boot.vfat): adding file 'bcm2710-rpi-3-b.dtb' as 'bcm2710-rpi-3-b.dtb' ...
vfat(boot.vfat): adding file 'bcm2837-rpi-3-b.dtb' as 'bcm2837-rpi-3-b.dtb' ...
vfat(boot.vfat): adding file 'rpi-firmware/bootcode.bin' as 'rpi-firmware/bootcode.bin' ...
vfat(boot.vfat): adding file 'rpi-firmware/cmdline.txt' as 'rpi-firmware/cmdline.txt' ...
vfat(boot.vfat): adding file 'rpi-firmware/config.txt' as 'rpi-firmware/config.txt' ...
vfat(boot.vfat): adding file 'rpi-firmware/fixup.dat' as 'rpi-firmware/fixup.dat' ...
vfat(boot.vfat): adding file 'rpi-firmware/start.elf' as 'rpi-firmware/start.elf' ...
vfat(boot.vfat): adding file 'Image' as 'Image' ...
hdimage(sdcard.img): adding partition 'boot' (in MBR) from 'boot.vfat' ...
hdimage(sdcard.img): adding partition 'rootfs' (in MBR) from 'rootfs.ext4' ...
hdimage(sdcard.img): writing MBR

What Linux distribution/version are you using?

-- 
Bye, Peter Korsgaard

[Buildroot] Buildroot 2018.05-rc3 released

[Peter Korsgaard]

>>>>> "Mike" == Mike Davies <busybox_mail@rovoreed.com> writes:

Please keep the discussion on the buildroot list, thanks.

 >> ?> If I select Python as a package to install on my target, then it
 >> builds
 >> ?> OK - but I don't want Python on my target.
 >> 
 >> ?> For now, I guess a workaround is to select Python, and delete it
 >> in a
 >> ?> pre-image script.
 >> 
 >> That is odd. The presence or not of python for the target shouldn't
 >> have
 >> any effect on host-python. You do not mean python3 for the target, do
 >> you?

 > Yes, sorry, I mean Python3 for the target. I also have ".pyc compiled
 > sources only", core python3 modules??--->?[*] unicodedata module, ?core
 > python3 modules??--->?(none)?

Ok, interesting. Enabling python3 for the target means that our host
python becomes python3, so the certdata2pem.py step in ca-certificates
is executed with host-python3.


>> What Linux distribution/version are you using?

 > The Linux distribution/version I am building on is Fedora Red Hat 25
 > ...

 > [mike at modron ~]$ uname -a
 > Linux modron 4.9.14-200.fc25.x86_64 #1 SMP Mon Mar 13 19:26:40 UTC 2017
 > x86_64 x86_64 x86_64 GNU/Linux

Ok, I'm on Debian. Thomas, you are using Fedora now, right? Can you give
ca-certificates a try?

-- 
Bye, Peter Korsgaard

[Buildroot] Buildroot 2018.05-rc3 released

[Thomas Petazzoni]

Hello,

On Wed, 30 May 2018 17:42:32 +0200, Peter Korsgaard wrote:

>  > Yes, sorry, I mean Python3 for the target. I also have ".pyc compiled
>  > sources only", core python3 modules??--->?[*] unicodedata module, ?core
>  > python3 modules??--->?(none)?  
> 
> Ok, interesting. Enabling python3 for the target means that our host
> python becomes python3,

Yes that is expected.

>  > The Linux distribution/version I am building on is Fedora Red Hat 25
>  > ...  
> 
>  > [mike at modron ~]$ uname -a
>  > Linux modron 4.9.14-200.fc25.x86_64 #1 SMP Mon Mar 13 19:26:40 UTC 2017
>  > x86_64 x86_64 x86_64 GNU/Linux  
> 
> Ok, I'm on Debian. Thomas, you are using Fedora now, right? Can you give
> ca-certificates a try?

It builds fine here: http://code.bulix.org/092xcg-342074?raw

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] Buildroot 2018.05-rc3 released

[Romain Naour]

Hi,

Le 30/05/2018 ? 18:00, Thomas Petazzoni a ?crit?:
> Hello,
> 
> On Wed, 30 May 2018 17:42:32 +0200, Peter Korsgaard wrote:
> 
>>  > Yes, sorry, I mean Python3 for the target. I also have ".pyc compiled
>>  > sources only", core python3 modules??--->?[*] unicodedata module, ?core
>>  > python3 modules??--->?(none)?  
>>
>> Ok, interesting. Enabling python3 for the target means that our host
>> python becomes python3,
> 
> Yes that is expected.
> 
>>  > The Linux distribution/version I am building on is Fedora Red Hat 25
>>  > ...  
>>
>>  > [mike at modron ~]$ uname -a
>>  > Linux modron 4.9.14-200.fc25.x86_64 #1 SMP Mon Mar 13 19:26:40 UTC 2017
>>  > x86_64 x86_64 x86_64 GNU/Linux  
>>
>> Ok, I'm on Debian. Thomas, you are using Fedora now, right? Can you give
>> ca-certificates a try?
> 
> It builds fine here: http://code.bulix.org/092xcg-342074?raw

The autobuilder at Smile is using Fedora 28, there is no issue with ca-certificates.

http://autobuild.buildroot.org/?submitter=Romain+Naour+(Smile)&&status=NOK

Romain

> 
> Thomas
> 

[Buildroot] Buildroot 2018.05-rc3 released

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

Hi,

 >> Ok, interesting. Enabling python3 for the target means that our host
 >> python becomes python3,

 > Yes that is expected.

Yes, but that could conceptually explain why Mike sees a difference in
behaviour.

 >> > The Linux distribution/version I am building on is Fedora Red Hat 25
 >> > ...  
 >> 
 >> > [mike at modron ~]$ uname -a
 >> > Linux modron 4.9.14-200.fc25.x86_64 #1 SMP Mon Mar 13 19:26:40 UTC 2017
 >> > x86_64 x86_64 x86_64 GNU/Linux  
 >> 
 >> Ok, I'm on Debian. Thomas, you are using Fedora now, right? Can you give
 >> ca-certificates a try?

 > It builds fine here: http://code.bulix.org/092xcg-342074?raw

Thanks, just like on my machine and the autobuilders.

Mike, there must be something odd going on with your machine. I'm afraid
you will have to debug it further yourself. Maybe try running
certdata2pem.py manually?

-- 
Bye, Peter Korsgaard

[Buildroot] Buildroot 2018.05-rc3 released

[Mike Davies]

On Wed, 2018-05-30 at 18:15 +0200, Peter Korsgaard wrote:
> > > > > > "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > > > > > writes:
> 
> Hi,
> 
> ?>> Ok, interesting. Enabling python3 for the target means that our
> host
> ?>> python becomes python3,
> 
> ?> Yes that is expected.
> 
> Yes, but that could conceptually explain why Mike sees a difference
> in
> behaviour.
> 
> ?>> > The Linux distribution/version I am building on is Fedora Red
> Hat 25
> ?>> > ...??
> ?>>?
> ?>> > [mike at modron ~]$ uname -a
> ?>> > Linux modron 4.9.14-200.fc25.x86_64 #1 SMP Mon Mar 13 19:26:40
> UTC 2017
> ?>> > x86_64 x86_64 x86_64 GNU/Linux??
> ?>>?
> ?>> Ok, I'm on Debian. Thomas, you are using Fedora now, right? Can
> you give
> ?>> ca-certificates a try?
> 
> ?> It builds fine here: http://code.bulix.org/092xcg-342074?raw
> 
> Thanks, just like on my machine and the autobuilders.
> 
> Mike, there must be something odd going on with your machine. I'm
> afraid
> you will have to debug it further yourself. Maybe try running
> certdata2pem.py manually?


OK. I shall try to debug this myself.

I have also noted the locale of my final built system is weird. Maybe
it's all related.

Mike

[Buildroot] Buildroot 2018.05-rc3 released

[Mike Davies]

On Wed, 2018-05-30 at 18:15 +0200, Peter Korsgaard wrote:

> 
> Mike, there must be something odd going on with your machine. I'm
> afraid
> you will have to debug it further yourself. Maybe try running
> certdata2pem.py manually?


OK, well, it seems to be that when host_python is being configured, the
configure script has not got the option...

--enable-unicode=ucs4

and it's not got that because python.mk uses a symbol
BR2_PACKAGE_PYTHON_UCS4 to decide whether to add that option to
HOST_PYTHON_CONF_OPTS

I do not have?BR2_PACKAGE_PYTHON_UCS4 in my .config file.

[mike at modron buildroot-2018.05-rc3]$ cat .config | grep PYTHON
# BR2_PACKAGE_MICROPYTHON is not set
# BR2_PACKAGE_HOST_PYTHON_CYTHON is not set
# BR2_PACKAGE_HOST_PYTHON_LXML is not set
# BR2_PACKAGE_HOST_PYTHON_SIX is not set
# BR2_PACKAGE_HOST_PYTHON_XLRD is not set
# BR2_PACKAGE_PYTHON_PYXML is not set
[mike at modron buildroot-2018.05-rc3]$?

I do not know the implications of simply moving --enable-unicode=ucs4
to the list of HOST_PYTHON_CONF_OPTS at the start of python.mk.

Mike

[Buildroot] Buildroot 2018.05-rc3 released

[Thomas Petazzoni]

Hello,

On Thu, 31 May 2018 15:52:49 +0100, Mike Davies wrote:

> OK, well, it seems to be that when host_python is being configured, the
> configure script has not got the option...
> 
> --enable-unicode=ucs4
> 
> and it's not got that because python.mk uses a symbol
> BR2_PACKAGE_PYTHON_UCS4 to decide whether to add that option to
> HOST_PYTHON_CONF_OPTS
> 
> I do not have?BR2_PACKAGE_PYTHON_UCS4 in my .config file.
> 
> [mike at modron buildroot-2018.05-rc3]$ cat .config | grep PYTHON
> # BR2_PACKAGE_MICROPYTHON is not set
> # BR2_PACKAGE_HOST_PYTHON_CYTHON is not set
> # BR2_PACKAGE_HOST_PYTHON_LXML is not set
> # BR2_PACKAGE_HOST_PYTHON_SIX is not set
> # BR2_PACKAGE_HOST_PYTHON_XLRD is not set
> # BR2_PACKAGE_PYTHON_PYXML is not set
> [mike at modron buildroot-2018.05-rc3]$?
> 
> I do not know the implications of simply moving --enable-unicode=ucs4
> to the list of HOST_PYTHON_CONF_OPTS at the start of python.mk.

But why would UCS4 support be required for you to build ca-certificates
and not on other systems ?

My test build had BR2_PACKAGE_PYTHON_UCS4 disabled, so
--enable-unicode=ucs4 was also not passed when building host-python.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] Buildroot 2018.05-rc3 released

[Mike Davies]

On Thu, 2018-05-31 at 17:26 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 31 May 2018 15:52:49 +0100, Mike Davies wrote:
> 
> > OK, well, it seems to be that when host_python is being configured,
> > the
> > configure script has not got the option...
> > 
> > --enable-unicode=ucs4
> > 
> > and it's not got that because python.mk uses a symbol
> > BR2_PACKAGE_PYTHON_UCS4 to decide whether to add that option to
> > HOST_PYTHON_CONF_OPTS
> > 
> > I do not have?BR2_PACKAGE_PYTHON_UCS4 in my .config file.
> > 
> > [mike at modron buildroot-2018.05-rc3]$ cat .config | grep PYTHON
> > # BR2_PACKAGE_MICROPYTHON is not set
> > # BR2_PACKAGE_HOST_PYTHON_CYTHON is not set
> > # BR2_PACKAGE_HOST_PYTHON_LXML is not set
> > # BR2_PACKAGE_HOST_PYTHON_SIX is not set
> > # BR2_PACKAGE_HOST_PYTHON_XLRD is not set
> > # BR2_PACKAGE_PYTHON_PYXML is not set
> > [mike at modron buildroot-2018.05-rc3]$?
> > 
> > I do not know the implications of simply moving --enable-
> > unicode=ucs4
> > to the list of HOST_PYTHON_CONF_OPTS at the start of python.mk.
> 
> But why would UCS4 support be required for you to build ca-
> certificates
> and not on other systems ?
> 
> My test build had BR2_PACKAGE_PYTHON_UCS4 disabled, so
> --enable-unicode=ucs4 was also not passed when building host-python.
> 

Why it builds OK for others and not me, then, sorry, I don't know.

When I build ca-certificates I saw that host-python was complaining
about undefined symbol: PyUnicodeUCS2_AsEncodedString, so I fiddled
about with the parameters to the host python configure script, noted
that if I force the host python configure with --enable-unicode=ucs4
then it works. If I change that to --enable-unicode=ucs2 or --enable-
unicode then it fails the same.

I'm not a Python programmer. This PC does not have Python on it at all,
and it only gets dragged in by buildroot to build ca-certificates.
Maybe that is something to do with it.

Let's not waste any more time on it.

Mike

[Buildroot] Buildroot 2018.05-rc3 released

[Arnout Vandecappelle]

On 31-05-18 20:30, Mike Davies wrote:
> On Thu, 2018-05-31 at 17:26 +0200, Thomas Petazzoni wrote:
>> Hello,
>>
>> On Thu, 31 May 2018 15:52:49 +0100, Mike Davies wrote:
[snip]
>>> I do not know the implications of simply moving --enable-
>>> unicode=ucs4
>>> to the list of HOST_PYTHON_CONF_OPTS at the start of python.mk.
>>
>> But why would UCS4 support be required for you to build ca-
>> certificates
>> and not on other systems ?
>>
>> My test build had BR2_PACKAGE_PYTHON_UCS4 disabled, so
>> --enable-unicode=ucs4 was also not passed when building host-python.
>>
> 
> Why it builds OK for others and not me, then, sorry, I don't know.
> 
> When I build ca-certificates I saw that host-python was complaining
> about undefined symbol: PyUnicodeUCS2_AsEncodedString, so I fiddled
> about with the parameters to the host python configure script, noted
> that if I force the host python configure with --enable-unicode=ucs4
> then it works. If I change that to --enable-unicode=ucs2 or --enable-
> unicode then it fails the same.
> 
> I'm not a Python programmer. This PC does not have Python on it at all,

 I doubt that: the dependencies.sh script checks that there is a "python"
executable in PATH, and the check-uniq-files script that is called at the end of
the build is also a Python script.

> and it only gets dragged in by buildroot to build ca-certificates.
> Maybe that is something to do with it.
> 
> Let's not waste any more time on it.

 Well, this may happen to other people as well, so it would be good to get to
the bottom of it.

 Let's see if we can find where the problem is. Can you do

readelf -d output/host/bin/python

readelf -W -s output/host/lib/libpython2.7.so | \
  grep PyUnicodeUCS._AsEncodedString

readelf -W -s output/build/host-python-2.7.14/Objects/unicodeobject.o | \
  grep PyUnicodeUCS._AsEncodedString



 Regards,
 Arnout

> 
> Mike
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Buildroot 2018.05-rc3 released

[Mike Davies]

On Thu, 2018-05-31 at 22:18 +0200, Arnout Vandecappelle wrote:
> 
> 
> ?Well, this may happen to other people as well, so it would be good
> to get to the bottom of it.
> 
> ?Let's see if we can find where the problem is. Can you do
> 
> readelf -d output/host/bin/python

[mike at modron buildroot-2018.05]$ readelf -d output/host/bin/python

Dynamic section at offset 0xdc0 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpython2.7.so.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libutil.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/mike/work/pi/buildroot-2018.05/output/host/lib]
 0x000000000000000c (INIT)               0x4006a8
 0x000000000000000d (FINI)               0x400864
 0x0000000000000019 (INIT_ARRAY)         0x600da8
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x600db0
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x400298
 0x0000000000000005 (STRTAB)             0x4004c0
 0x0000000000000006 (SYMTAB)             0x4002f8
 0x000000000000000a (STRSZ)              345 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x601000
 0x0000000000000002 (PLTRELSZ)           24 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x400690
 0x0000000000000007 (RELA)               0x400660
 0x0000000000000008 (RELASZ)             48 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x400640
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x40061a
 0x0000000000000000 (NULL)               0x0
[mike at modron buildroot-2018.05]$ 

> readelf -W -s output/host/lib/libpython2.7.so | \
> ? grep PyUnicodeUCS._AsEncodedString


[mike at modron buildroot-2018.05]$ readelf -W -s output/host/lib/libpython2.7.so | grep PyUnicodeUCS._AsEncodedString
  1374: 00000000000dcbe0   308 FUNC    GLOBAL DEFAULT   12 PyUnicodeUCS2_AsEncodedString
  4961: 00000000000dcbe0   308 FUNC    GLOBAL DEFAULT   12 PyUnicodeUCS2_AsEncodedString
[mike at modron buildroot-2018.05]$ 


> readelf -W -s output/build/host-python-2.7.14/Objects/unicodeobject.o 
> | \
> ? grep PyUnicodeUCS._AsEncodedString
> 

[mike at modron buildroot-2018.05]$ readelf -W -s output/build/host-python-2.7.14/Objects/unicodeobject.o | grep PyUnicodeUCS._AsEncodedString
   482: 000000000000f680   308 FUNC    GLOBAL DEFAULT    1 PyUnicodeUCS2_AsEncodedString
[mike at modron buildroot-2018.05]$ 


Now, if I understand these dumps correctly, the symbol is there, but
this new release still fails to build ca-certificates ...

>>> ca-certificates 20170717 Patching
>>> ca-certificates 20170717 Configuring
>>> ca-certificates 20170717 Building
PATH="/home/mike/work/pi/buildroot-2018.05/output/host/bin:/home/mike/work/pi/buildroot-2018.05/output/host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/mike/bin:/sbin:/home/mike/.local/bin:/home/mike/bin:/home/mike/bin:/sbin" /usr/bin/make -j9 -C /home/mike/work/pi/buildroot-2018.05/output/build/ca-certificates-20170717 all
for dir in mozilla; do \
  /usr/bin/make -C $dir all; \
done
python certdata2pem.py
Traceback (most recent call last):
  File "certdata2pem.py", line 23, in <module>
    import base64
  File "/home/mike/work/pi/buildroot-2018.05/output/host/lib64/python2.7/base64.py", line 9, in <module>
    import struct
  File "/home/mike/work/pi/buildroot-2018.05/output/host/lib64/python2.7/struct.py", line 1, in <module>
    from _struct import *
ImportError: /home/mike/work/pi/buildroot-2018.05/output/host/lib64/python2.7/lib-dynload/_struct.so: undefined symbol: PyUnicodeUCS2_AsEncodedString
Makefile:6: recipe for target 'all' failed
make[3]: *** [all] Error 1
Makefile:9: recipe for target 'all' failed
make[2]: *** [all] Error 2
package/pkg-generic.mk:223: recipe for target '/home/mike/work/pi/buildroot-2018.05/output/build/ca-certificates-20170717/.stamp_built' failed
make[1]: *** [/home/mike/work/pi/buildroot-2018.05/output/build/ca-certificates-20170717/.stamp_built] Error 2
Makefile:79: recipe for target '_all' failed
make: *** [_all] Error 2
[mike at modron buildroot-2018.05]$ 

I hope this is of some help, because I'm even more confused now. The
symbol really is undefined in _struct.so, so why isn't it picking it up
from libpython.2.7.so, because if I understand the following dumps it
should.

[mike at modron buildroot-2018.05]$ readelf -d output/host/lib64/python2.7/lib-dynload/_struct.so

Dynamic section at offset 0x8db8 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpython2.7.so.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/mike/work/pi/buildroot-2018.05/output/host/lib]
 0x000000000000000c (INIT)               0x2538
 0x000000000000000d (FINI)               0x74e4
 0x0000000000000019 (INIT_ARRAY)         0x208d98
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x208da0
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x1f0
 0x0000000000000005 (STRTAB)             0xa28
 0x0000000000000006 (SYMTAB)             0x230
 0x000000000000000a (STRSZ)              1533 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x209000
 0x0000000000000002 (PLTRELSZ)           1488 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x1f68
 0x0000000000000007 (RELA)               0x1110
 0x0000000000000008 (RELASZ)             3672 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x10d0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x1026
 0x000000006ffffff9 (RELACOUNT)          138
 0x0000000000000000 (NULL)               0x0
[mike at modron buildroot-2018.05]$ readelf -W -s output/host/lib64/python2.7/lib-dynload/_struct.so | grep PyUnicodeUCS._AsEncodedString
    62: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND PyUnicodeUCS2_AsEncodedString
   217: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND PyUnicodeUCS2_AsEncodedString
[mike at modron buildroot-2018.05]$ 

Mike