Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]

Hi,

I recently was able to get a master checkout of OECore + meta-ti to build
my custom board kernel using linux-ti81xx-psp_2.6.37.bb.  I tried to make
the same modifications to a master yocto + meta-ti checkout and when I try
to build, bitbake absolutely takes forever to build
linux-ti81xx-psp_2.6.37.bb and it gets stuck on the do_package.  I finally
just let it go (hours and hours on a very fast modern machine) and it looks
like it finally just gave up the ghost and ran out of memory or something.
I've never seen anything like it.  Anyone seen anything like this before:

Build Configuration:
BB_VERSION        = "1.20.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Debian-7.1"
TARGET_SYS        = "arm-poky-linux-gnueabi"
MACHINE           = "c6a816x-evm"
DISTRO            = "poky"
DISTRO_VERSION    = "1.5"
TUNE_FEATURES     = "armv7a vfp neon"
TARGET_FPU        = "vfp-neon"
meta
meta-yocto
meta-yocto-bsp    = "master:104166c804e9068d85ccdaa887a33df72c8aeb85"
meta-ti           = "master:9c36e6c7573c3e68c5816dd04f3dd43f485b0550"
meta-oe
meta-networking
meta-webserver    = "master:8aafddccb03f54923927e9d1c8c80223baf581de"

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: Error executing a python function in
/home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
linux-ti81xx-psp_2.6.37.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'package_do_filedeps', lineno: 56, function:
<module>

     0052:    for pkg in provides_files:
     0053:        d.setVar("FILERPROVIDESFLIST_" + pkg, "
".join(provides_files[pkg]))
     0054:
     0055:
 *** 0056:package_do_filedeps(d)
     0057:
File: 'package_do_filedeps', lineno: 25, function: package_do_filedeps
     0021:
     0022:    import multiprocessing
     0023:    nproc = multiprocessing.cpu_count()
     0024:    pool =  bb.utils.multiprocessingpool(nproc)
 *** 0025:    processed = list(pool.imap(oe.package.filedeprunner, pkglist))
     0026:    pool.close()
     0027:    pool.join()
     0028:
     0029:    provides_files = {}
File: '/usr/lib/python2.7/multiprocessing/pool.py', lineno: 653, function:
next
     0649:
     0650:        success, value = item
     0651:        if success:
     0652:            return value
 *** 0653:        raise value
     0654:
     0655:    __next__ = next                    # XXX
     0656:
     0657:    def _set(self, i, obj):
Exception: OSError: [Errno 12] Cannot allocate memory

ERROR: Function failed: package_do_filedeps
ERROR: Logfile of failure stored in:
/home/hutch/yocto_master_wcs_10_4_2013/poky/build/tmp/work/c6a816x_evm-poky-linux-gnueabi/linux-ti81xx-psp/2.6.37-r0/temp/log.do_package.12766
ERROR: Task 200
(/home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
linux-ti81xx-psp_2.6.37.bb, do_package) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2756 tasks of which 2752 didn't need to be
rerun and 1 failed.
No currently running tasks (2023 of 2770)

Summary: 1 task failed:
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
linux-ti81xx-psp_2.6.37.bb, do_package
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

[-- Attachment #2: Type: text/html, Size: 4091 bytes --]

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Denys Dmytriyenko]

On Wed, Oct 09, 2013 at 09:10:49AM -0400, Brian Hutchinson wrote:
> Hi,
> 
> I recently was able to get a master checkout of OECore + meta-ti to build
> my custom board kernel using linux-ti81xx-psp_2.6.37.bb.  I tried to make
> the same modifications to a master yocto + meta-ti checkout and when I try
> to build, bitbake absolutely takes forever to build

I have 2 questions:

1. Which bitbake you use when building oe-core+meta-ti? Since Poky comes with 
own copy of bitbake, you may want to compare them.

2. Not that it would affect packaging, but where in bblayer.conf did you add 
meta-ti?

And one more suggestion - you may want to compare classes between oe-core and 
poky/meta to see if there are any big differences, especially in the packaging 
area...

-- 
Denys


> linux-ti81xx-psp_2.6.37.bb and it gets stuck on the do_package.  I finally
> just let it go (hours and hours on a very fast modern machine) and it looks
> like it finally just gave up the ghost and ran out of memory or something.
> I've never seen anything like it.  Anyone seen anything like this before:
> 
> Build Configuration:
> BB_VERSION        = "1.20.0"
> BUILD_SYS         = "x86_64-linux"
> NATIVELSBSTRING   = "Debian-7.1"
> TARGET_SYS        = "arm-poky-linux-gnueabi"
> MACHINE           = "c6a816x-evm"
> DISTRO            = "poky"
> DISTRO_VERSION    = "1.5"
> TUNE_FEATURES     = "armv7a vfp neon"
> TARGET_FPU        = "vfp-neon"
> meta
> meta-yocto
> meta-yocto-bsp    = "master:104166c804e9068d85ccdaa887a33df72c8aeb85"
> meta-ti           = "master:9c36e6c7573c3e68c5816dd04f3dd43f485b0550"
> meta-oe
> meta-networking
> meta-webserver    = "master:8aafddccb03f54923927e9d1c8c80223baf581de"
> 
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: Executing RunQueue Tasks
> ERROR: Error executing a python function in
> /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
> linux-ti81xx-psp_2.6.37.bb:
> 
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'package_do_filedeps', lineno: 56, function:
> <module>
> 
>      0052:    for pkg in provides_files:
>      0053:        d.setVar("FILERPROVIDESFLIST_" + pkg, "
> ".join(provides_files[pkg]))
>      0054:
>      0055:
>  *** 0056:package_do_filedeps(d)
>      0057:
> File: 'package_do_filedeps', lineno: 25, function: package_do_filedeps
>      0021:
>      0022:    import multiprocessing
>      0023:    nproc = multiprocessing.cpu_count()
>      0024:    pool =  bb.utils.multiprocessingpool(nproc)
>  *** 0025:    processed = list(pool.imap(oe.package.filedeprunner, pkglist))
>      0026:    pool.close()
>      0027:    pool.join()
>      0028:
>      0029:    provides_files = {}
> File: '/usr/lib/python2.7/multiprocessing/pool.py', lineno: 653, function:
> next
>      0649:
>      0650:        success, value = item
>      0651:        if success:
>      0652:            return value
>  *** 0653:        raise value
>      0654:
>      0655:    __next__ = next                    # XXX
>      0656:
>      0657:    def _set(self, i, obj):
> Exception: OSError: [Errno 12] Cannot allocate memory
> 
> ERROR: Function failed: package_do_filedeps
> ERROR: Logfile of failure stored in:
> /home/hutch/yocto_master_wcs_10_4_2013/poky/build/tmp/work/c6a816x_evm-poky-linux-gnueabi/linux-ti81xx-psp/2.6.37-r0/temp/log.do_package.12766
> ERROR: Task 200
> (/home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
> linux-ti81xx-psp_2.6.37.bb, do_package) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 2756 tasks of which 2752 didn't need to be
> rerun and 1 failed.
> No currently running tasks (2023 of 2770)
> 
> Summary: 1 task failed:
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux/
> linux-ti81xx-psp_2.6.37.bb, do_package
> Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

> _______________________________________________
> meta-ti mailing list
> meta-ti@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-ti

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 3622 bytes --]

On Wed, Oct 9, 2013 at 2:09 PM, Denys Dmytriyenko <denys@ti.com> wrote:

> I have 2 questions:
>
> 1. Which bitbake you use when building oe-core+meta-ti? Since Poky comes
> with
> own copy of bitbake, you may want to compare them.
>
> 2. Not that it would affect packaging, but where in bblayer.conf did you
> add
> meta-ti?
>
> And one more suggestion - you may want to compare classes between oe-core
> and
> poky/meta to see if there are any big differences, especially in the
> packaging
> area...
>
> --
> Denys
>

Hey Denys,

I'll look up the versions here in a sec.  I'm about to kill a build that
has been running for about 3hrs stuck again on just:

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
Currently 1 running tasks (1633 of 2770):
0: linux-ti81xx-psp-2.6.37-r0 do_package (pid 11284)

I researched some of the error messages I got last time and discovered the
SKIP_FILEDEPS flag so I put that in my linux-ti81xx-psp_2.6.37.bb recipe to
see if that would maybe make a difference.  So far when I look at
log.do_package it has done:

DEBUG: Executing python function sstate_task_prefunc
DEBUG: Python function sstate_task_prefunc finished
DEBUG: Executing python function do_package
DEBUG: Executing python function package_get_auto_pr
DEBUG: Python function package_get_auto_pr finished
DEBUG: Executing python function perform_packagecopy
DEBUG: Python function perform_packagecopy finished
DEBUG: Executing python function split_and_strip_files
DEBUG: Python function split_and_strip_files finished
DEBUG: Executing python function fixup_perms
DEBUG: Python function fixup_perms finished
DEBUG: Executing python function split_kernel_packages
DEBUG: Python function split_kernel_packages finished
DEBUG: Executing python function split_kernel_module_packages
DEBUG: Python function split_kernel_module_packages finished
DEBUG: Executing python function package_do_split_locales
DEBUG: No locale files in this package
DEBUG: Python function package_do_split_locales finished
DEBUG: Executing python function populate_packages

... but it has been stuck there for quite a while and I think something is
probably dead again.  I do a ps and see some decafbad so that probably
isn't good ... and I've been getting plenty of caffeine today :)

hutch@neo:~/yocto_master_wcs_10_4_2013/poky/meta-ti/recipes-kernel/linux$
ps aux | grep -i wcs
hutch    11198  0.0  1.0 140408 82788 pts/3    S+   11:02   0:08 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake
core-image-wcs
hutch    11199  0.0  0.1 179204 14960 pts/3    Sl+  11:02   0:00 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake
core-image-wcs
hutch    11206  3.0  3.7 414052 305136 pts/3   Sl+  11:02   6:08 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake
core-image-wcs
hutch    11280  0.0  1.5 161308 123588 pts/3   S+   11:03   0:02 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake-worker
decafbad
hutch    11282  0.0  1.5 163604 123760 pts/3   S+   11:03   0:03 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake-worker
decafbad
hutch    11284 83.2 13.6 1163748 1121848 pts/3 R    11:03 169:13 python
/home/hutch/yocto_master_wcs_10_4_2013/poky/bitbake/bin/bitbake-worker
decafbad
hutch    11286  9.9 13.7 1138824 1130740 ?     Ss   11:03  20:09
/home/hutch/yocto_master_wcs_10_4_2013/poky/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo
-d
hutch    11696  0.0  0.0   7828   860 pts/20   S+   14:26   0:00 grep -i wcs

[-- Attachment #2: Type: text/html, Size: 4268 bytes --]

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]

On Wed, Oct 9, 2013 at 2:09 PM, Denys Dmytriyenko <denys@ti.com> wrote:

>
> 1. Which bitbake you use when building oe-core+meta-ti? Since Poky comes
> with
> own copy of bitbake, you may want to compare them.
>

Poky is: BitBake Build Tool Core version 1.20.0, bitbake version 1.20.0
OECore is:  BitBake Build Tool Core version 1.19.1, bitbake version 1.19.1


> 2. Not that it would affect packaging, but where in bblayer.conf did you
> add
> meta-ti?
>

My bblayers.conf in Poky is:
BBLAYERS ?= " \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto-bsp \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti \
  /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-oe \
  /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-networking \
  /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-webserver \
  "
BBLAYERS_NON_REMOVABLE ?= " \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta \
  /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto \

My bblayers.conf in OECore (that works) is:
BBLAYERS ?= " \
  /home/hutch/oe_core_beagleboard/oe-core/meta \
  /home/hutch/oe_core_beagleboard/oe-core/meta-ti \
  /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-oe \
  /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-networking
\
  /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-webserver \
  "

BBLAYERS_NON_REMOVABLE ?= " \
  /home/hutch/oe_core_beagleboard/oe-core/meta \
  "

I killed the build.  I can't afford to tie up the server that long.  Looks
like I'll have to go back to using Poky for only building my rootfs and
build kernel with poky generated sdk outside of bitbake.  The other problem
I saw was in OECore, I added the linux-libc-headers for 2.6.37.2 so
packages would be built with headers that matched my TI PSP based kernel
and I was able to get that to work.  In Poky, I made the same mods to try
and build everything against 2.6.37.2 headers and no matter what I tried it
would always use 3.10 headers.  Probably OK but I still worry it could
cause a problem.


> And one more suggestion - you may want to compare classes between oe-core
> and
> poky/meta to see if there are any big differences, especially in the
> packaging
> area...
>

Will do.  Thanks!

Brian

[-- Attachment #2: Type: text/html, Size: 3242 bytes --]

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Wed, Oct 9, 2013 at 3:15 PM, Brian Hutchinson <b.hutchman@gmail.com>wrote:

> ...  The other problem I saw was in OECore, I added the linux-libc-headers
> for 2.6.37.2 so packages would be built with headers that matched my TI PSP
> based kernel and I was able to get that to work.  In Poky, I made the same
> mods to try and build everything against 2.6.37.2 headers and no matter
> what I tried it would always use 3.10 headers.  Probably OK but I still
> worry it could cause a problem.
>
>
I retract the statement I made earlier regarding not being able to use
2.6.37.2 libc-headers in Poky.  Looks like I didn't have the recipe setup
quite right.  I studied the differences some more between how OECore and
Poky are set up and made more mods to the recipes and now it looks like it
is fetching and attempting to use the version I asked for (doing a
beaglebone build instead of c6a816x-evm build this time).

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 1370 bytes --]

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Denys Dmytriyenko]

On Wed, Oct 09, 2013 at 03:15:17PM -0400, Brian Hutchinson wrote:
> On Wed, Oct 9, 2013 at 2:09 PM, Denys Dmytriyenko <denys@ti.com> wrote:
> 
> >
> > 1. Which bitbake you use when building oe-core+meta-ti? Since Poky comes
> > with
> > own copy of bitbake, you may want to compare them.
> >
> 
> Poky is: BitBake Build Tool Core version 1.20.0, bitbake version 1.20.0
> OECore is:  BitBake Build Tool Core version 1.19.1, bitbake version 1.19.1

Have you tried matching them up? If you update bitbake in oe-core to 1.20 will 
it hang too?


> > 2. Not that it would affect packaging, but where in bblayer.conf did you
> > add
> > meta-ti?
> >
> 
> My bblayers.conf in Poky is:
> BBLAYERS ?= " \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto-bsp \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-ti \
>   /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-oe \
>   /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-networking \
>   /home/hutch/yocto_master_wcs_10_4_2013/layers/meta-oe/meta-webserver \

I'm wondering who's insisting on this somewhat upside-down order of layers in 
the stack? Unfortunately, layer priority is not the only mechanism to control 
parsing order... You may find this thread useful"

http://thread.gmane.org/gmane.linux.embedded.yocto.meta-ti/2726/focus=2744


>   "
> BBLAYERS_NON_REMOVABLE ?= " \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta \
>   /home/hutch/yocto_master_wcs_10_4_2013/poky/meta-yocto \
> 
> My bblayers.conf in OECore (that works) is:
> BBLAYERS ?= " \
>   /home/hutch/oe_core_beagleboard/oe-core/meta \
>   /home/hutch/oe_core_beagleboard/oe-core/meta-ti \
>   /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-oe \
>   /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-networking \
>   /home/hutch/oe_core_beagleboard/oe-core/meta-openembedded/meta-webserver \

Is it distro-less?


>   "
> 
> BBLAYERS_NON_REMOVABLE ?= " \
>   /home/hutch/oe_core_beagleboard/oe-core/meta \
>   "
> 
> I killed the build.  I can't afford to tie up the server that long.  Looks
> like I'll have to go back to using Poky for only building my rootfs and
> build kernel with poky generated sdk outside of bitbake.  The other problem
> I saw was in OECore, I added the linux-libc-headers for 2.6.37.2 so
> packages would be built with headers that matched my TI PSP based kernel
> and I was able to get that to work.  In Poky, I made the same mods to try
> and build everything against 2.6.37.2 headers and no matter what I tried it
> would always use 3.10 headers.  Probably OK but I still worry it could
> cause a problem.
> 
> 
> > And one more suggestion - you may want to compare classes between oe-core
> > and
> > poky/meta to see if there are any big differences, especially in the
> > packaging
> > area...
> >
> 
> Will do.  Thanks!

Any updates?

-- 
Denys

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Denys Dmytriyenko]

On Wed, Oct 09, 2013 at 03:15:17PM -0400, Brian Hutchinson wrote:
> The other problem I saw was in OECore, I added the linux-libc-headers for 
> 2.6.37.2 so packages would be built with headers that matched my TI PSP 
> based kernel and I was able to get that to work.

BTW, we do have it in meta-arago-extras, if you want to use it:

http://arago-project.org/git/?p=meta-arago.git;a=blob;f=meta-arago-extras/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.37.2.bb;hb=HEAD

-- 
Denys

Re: Kernel recipe that works with OECore + meta-ti fails with Poky + meta-ti

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

On Sun, Oct 13, 2013 at 8:20 PM, Denys Dmytriyenko <denys@ti.com> wrote:

> On Wed, Oct 09, 2013 at 03:15:17PM -0400, Brian Hutchinson wrote:
> > The other problem I saw was in OECore, I added the linux-libc-headers for
> > 2.6.37.2 so packages would be built with headers that matched my TI PSP
> > based kernel and I was able to get that to work.
>
> BTW, we do have it in meta-arago-extras, if you want to use it:
>
>
> http://arago-project.org/git/?p=meta-arago.git;a=blob;f=meta-arago-extras/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.37.2.bb;hb=HEAD
>

I stole it from somewhere.  It was Khem & Hillman's  patch I believe.

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 1259 bytes --]