[PATCH v2 0/1] Resync siteinfo.bbclass with oe.dev

[PATCH v2 0/1] Resync siteinfo.bbclass with oe.dev

[Tom Rini]

Hey all,

This is the second of two pull requests.  This doesn't depend on the first
request, it's just splitting changes up.  There's three changes here.  First,
site/ix86-common was setting the wrong value for unsigned int.  Second,
change perl to use SITEINFO variables rather than functions.  
We're down to one change now to re-sync siteinfo.bbclass with oe-core
and oe.dev.  This re-works the functions a bit inside siteinfo.bbclass
to be easier to add new platforms (and hopefully multilibs) to.

My next step is to work on merging and verifying as much as possible
the site files in oe-core and oe.dev.

v2: Rework based on Richard's comments to not be a generator and just
return a string list again.

The following changes since commit 0d03800caffc2341b74190d38f7a372223c89f4c:
  Mei Lei (1):
        lsbsetup_1.0.bb: Change the hardcoded /usr/lib to support multilib

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib trini/update-siteinfo-bbclass-v2
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/update-siteinfo-bbclass-v2

Tom Rini (1):
  siteinfo.bbclass: Port over oe.dev logic for site files

 meta/classes/siteinfo.bbclass |  203 ++++++++++++++++++++---------------------
 1 files changed, 98 insertions(+), 105 deletions(-)
 create mode 100644 meta/site/common-linux

[PATCH v2 1/1] siteinfo.bbclass: Port over oe.dev logic for site files

[Tom Rini]

In oe.dev we have a sets to pick out hostos/hostarch/etc site
files out of and include things like a common-linux site file.
This should also help out with adding multilib-specific site files
(ie x32).  In oe.dev we have an iterator but at Richard's request
we continue to return a list of files in siteinfo_get_files().

Signed-off-by: Tom Rini <tom_rini@mentor.com>
---
 meta/classes/siteinfo.bbclass |  203 ++++++++++++++++++++---------------------
 1 files changed, 98 insertions(+), 105 deletions(-)
 create mode 100644 meta/site/common-linux

diff --git a/meta/classes/siteinfo.bbclass b/meta/classes/siteinfo.bbclass
index 78b7008..ff9fd0d 100644
--- a/meta/classes/siteinfo.bbclass
+++ b/meta/classes/siteinfo.bbclass
@@ -15,121 +15,114 @@
 # It is an error for the target not to exist.
 # If 'what' doesn't exist then an empty value is returned
 #
-def get_siteinfo_list(d):
-       target = bb.data.getVar('HOST_ARCH', d, 1) + "-" + bb.data.getVar('HOST_OS', d, 1)
+def siteinfo_data(d):
+    archinfo = {
+        "allarch": "endian-little bit-32", # bogus, but better than special-casing the checks below for allarch
+        "arm": "endian-little bit-32 arm-common",
+        "armeb": "endian-big bit-32 arm-common",
+        "avr32": "endian-big bit-32 avr32-common",
+        "bfin": "endian-little bit-32 bfin-common",
+        "i386": "endian-little bit-32 ix86-common",
+        "i486": "endian-little bit-32 ix86-common",
+        "i586": "endian-little bit-32 ix86-common",
+        "i686": "endian-little bit-32 ix86-common",
+        "ia64": "endian-little bit-64",
+        "microblaze": "endian-big bit-32 microblaze-common",
+        "microblazeel": "endian-little bit-32 microblaze-common",
+        "mips": "endian-big bit-32 mips-common",
+        "mips64": "endian-big bit-64 mips64-common",
+        "mips64el": "endian-little bit-64 mips64-common",
+        "mipsel": "endian-little bit-32 mips-common",
+        "powerpc": "endian-big bit-32 powerpc-common",
+        "nios2": "endian-little bit-32 nios2-common",
+        "powerpc64": "endian-big bit-64 powerpc-common powerpc64-linux",
+        "ppc": "endian-big bit-32 powerpc-common",
+        "ppc64": "endian-big bit-64 powerpc-common powerpc64-linux",
+        "sh3": "endian-little bit-32 sh-common",
+        "sh4": "endian-little bit-32 sh-common",
+        "sparc": "endian-big bit-32",
+        "viac3": "endian-little bit-32 ix86-common",
+        "x86_64": "endian-little bit-64",
+    }
+    osinfo = {
+        "darwin": "common-darwin",
+        "darwin9": "common-darwin",
+        "linux": "common-linux common-glibc",
+        "linux-gnueabi": "common-linux common-glibc",
+        "linux-gnuspe": "common-linux common-glibc",
+        "linux-uclibc": "common-linux common-uclibc",
+        "linux-uclibceabi": "common-linux common-uclibc",
+        "linux-uclibcspe": "common-linux common-uclibc",
+        "uclinux-uclibc": "common-uclibc",
+        "cygwin": "common-cygwin",
+        "mingw32": "common-mingw",
+    }
+    targetinfo = {
+        "arm-linux-gnueabi": "arm-linux",
+        "arm-linux-uclibceabi": "arm-linux-uclibc",
+        "armeb-linux-gnueabi": "armeb-linux",
+        "armeb-linux-uclibceabi": "armeb-linux-uclibc",
+        "powerpc-linux-gnuspe": "powerpc-linux",
+        "powerpc-linux-uclibcspe": "powerpc-linux-uclibc",
+    }
 
-       targetinfo = {\
-               "allarch-linux":           "",\
-               "armeb-linux":             "endian-big bit-32 common-glibc arm-common",\
-               "armeb-linux-gnueabi":     "endian-big bit-32 common-glibc arm-common armeb-linux",\
-               "armeb-linux-uclibc":      "endian-big bit-32 common-uclibc arm-common",\
-               "armeb-linux-uclibceabi": "endian-big bit-32 common-uclibc arm-common armeb-linux-uclibc",\
-               "arm-darwin":              "endian-little bit-32 common-darwin",\
-               "arm-darwin8":              "endian-little bit-32 common-darwin",\
-               "arm-linux":               "endian-little bit-32 common-glibc arm-common",\
-               "arm-linux-gnueabi":       "endian-little bit-32 common-glibc arm-common arm-linux",\
-               "arm-linux-uclibc":        "endian-little bit-32 common-uclibc arm-common",\
-               "arm-linux-uclibceabi": "endian-little bit-32 common-uclibc arm-common arm-linux-uclibc",\
-               "avr32-linux":             "endian-big bit-32 common-glibc avr32-common",\ 
-               "avr32-linux-uclibc":      "endian-big bit-32 common-uclibc avr32-common",\
-               "bfin-uclinux-uclibc":       "endian-little bit-32 common-uclibc bfin-common",\
-               "i386-linux":              "endian-little bit-32 common-glibc ix86-common",\
-               "i486-linux":              "endian-little bit-32 common-glibc ix86-common",\
-               "i586-linux":              "endian-little bit-32 common-glibc ix86-common",\
-               "i686-linux":              "endian-little bit-32 common-glibc ix86-common",\
-               "i386-linux-uclibc":       "endian-little bit-32 common-uclibc ix86-common",\
-               "i486-linux-uclibc":       "endian-little bit-32 common-uclibc ix86-common",\
-               "i586-linux-uclibc":       "endian-little bit-32 common-uclibc ix86-common",\
-               "i686-linux-uclibc":       "endian-little bit-32 common-uclibc ix86-common",\
-               "microblaze-linux-gnu":    "endian-big bit-32 common-glibc microblaze-common",\
-               "microblazeel-linux-gnu":  "endian-little bit-32 common-glibc microblaze-common",\
-               "mipsel-linux":            "endian-little bit-32 common-glibc mips-common",\
-               "mipsel-linux-uclibc":     "endian-little bit-32 common-uclibc mips-common",\
-               "mips-linux":              "endian-big bit-32 common-glibc mips-common",\
-               "mips-linux-uclibc":       "endian-big bit-32 common-uclibc mips-common",\
-               "powerpc-darwin":          "endian-big bit-32 common-darwin",\
-               "ppc-linux":               "endian-big bit-32 common-glibc powerpc-common",\ 
-	       "powerpc-linux":           "endian-big bit-32 common-glibc powerpc-common",\
-               "powerpc-linux-gnuspe":    "endian-big bit-32 common-glibc powerpc-common",\
-               "powerpc-linux-uclibc":    "endian-big bit-32 common-uclibc powerpc-common",\
-               "sh3-linux":               "endian-little bit-32 common-glibc sh-common",\
-               "sh4-linux":               "endian-little bit-32 common-glibc sh-common",\
-               "sh4-linux-uclibc":        "endian-little bit-32 common-uclibc sh-common",\
-               "sparc-linux":             "endian-big bit-32 common-glibc",\
-               "x86_64-linux":            "endian-little bit-64 common-glibc",\
-               "x86_64-linux-uclibc":     "endian-little bit-64 common-uclibc"}
-       if target in targetinfo:
-               info = targetinfo[target].split()
-               info.append(target)
-               info.append("common")
-               return info
-       else:
-               bb.error("Information not available for target '%s'" % target)
+    hostarch = d.getVar("HOST_ARCH", True)
+    hostos = d.getVar("HOST_OS", True)
+    target = "%s-%s" % (hostarch, hostos)
 
+    sitedata = []
+    if hostarch in archinfo:
+        sitedata.extend(archinfo[hostarch].split())
+    if hostos in osinfo:
+        sitedata.extend(osinfo[hostos].split())
+    if target in targetinfo:
+        sitedata.extend(targetinfo[target].split())
+    sitedata.append(target)
+    sitedata.append("common")
 
-#
-# Define which site files to use. We check for several site files and
-# use each one that is found, based on the list returned by get_siteinfo_list()
-#
-# Search for the files in the following directories:
-# 1) ${BBPATH}/site (in reverse) - app specific, then site wide
-# 2) ${FILE_DIRNAME}/site-${PV}         - app version specific
-#
-def siteinfo_get_files(d):
-       sitefiles = ""
+    bb.debug(1, "SITE files %s" % sitedata);
+    return sitedata
 
-       # Determine which site files to look for
-       sites = get_siteinfo_list(d)
+python () {
+    sitedata = set(siteinfo_data(d))
+    if "endian-little" in sitedata:
+        d.setVar("SITEINFO_ENDIANESS", "le")
+    elif "endian-big" in sitedata:
+        d.setVar("SITEINFO_ENDIANESS", "be")
+    else:
+        bb.error("Unable to determine endianness for architecture '%s'" %
+                 d.getVar("HOST_ARCH", True))
+        bb.fatal("Please add your architecture to siteinfo.bbclass")
 
-       # Check along bbpath for site files and append in reverse order so
-       # the application specific sites files are last and system site
-       # files first.
-       path_bb = bb.data.getVar('BBPATH', d, 1)
-       for p in (path_bb or "").split(':'):
-               tmp = ""
-               for i in sites:
-                       fname = os.path.join(p, 'site', i)
-                       if os.path.exists(fname):
-                               tmp += fname + " "
-               sitefiles = tmp + sitefiles;
+    if "bit-32" in sitedata:
+        d.setVar("SITEINFO_BITS", "32")
+    elif "bit-64" in sitedata:
+        d.setVar("SITEINFO_BITS", "64")
+    else:
+        bb.error("Unable to determine bit size for architecture '%s'" %
+                 d.getVar("HOST_ARCH", True))
+        bb.fatal("Please add your architecture to siteinfo.bbclass")
+}
 
-       # Now check for the applications version specific site files
-       path_pkgv = os.path.join(bb.data.getVar('FILE_DIRNAME', d, 1), "site-" + bb.data.getVar('PV', d, 1))
-       for i in sites:
-               fname = os.path.join(path_pkgv, i)
-               if os.path.exists(fname):
-                       sitefiles += fname + " "
-
-       # Now check for siteconfig cache files
-       path_siteconfig = bb.data.getVar('SITECONFIG_SYSROOTCACHE', d, 1)
-       if os.path.isdir(path_siteconfig):
-               for i in os.listdir(path_siteconfig):
-                       fname = os.path.join(path_siteconfig, i)
-                       sitefiles += fname + " "
-
-       bb.debug(1, "SITE files " + sitefiles);
-       return sitefiles
+def siteinfo_get_files(d):
+    sitedata = siteinfo_data(d)
+    sitefiles = ""
+    for path in d.getVar("BBPATH", True).split(":"):
+        for element in sitedata:
+            filename = os.path.join(path, "site", element)
+            if os.path.exists(filename):
+                sitefiles += filename + " "
 
-def siteinfo_get_endianess(d):
-       info = get_siteinfo_list(d)
-       if 'endian-little' in info:
-              return "le"
-       elif 'endian-big' in info:
-              return "be"
-       bb.error("Site info could not determine endianess for target")
+    # Now check for siteconfig cache files
+    path_siteconfig = bb.data.getVar('SITECONFIG_SYSROOTCACHE', d, 1)
+    if os.path.isdir(path_siteconfig):
+        for i in os.listdir(path_siteconfig):
+            filename = os.path.join(path_siteconfig, i)
+            sitefiles += filename + " "
 
-def siteinfo_get_bits(d):
-       info = get_siteinfo_list(d)
-       if 'bit-32' in info:
-              return "32"
-       elif 'bit-64' in info:
-              return "64"
-       bb.error("Site info could not determine bit size for target")
+    return sitefiles
 
 #
 # Make some information available via variables
 #
-SITEINFO_ENDIANESS  = "${@siteinfo_get_endianess(d)}"
-SITEINFO_BITS       = "${@siteinfo_get_bits(d)}"
 SITECONFIG_SYSROOTCACHE = "${STAGING_DATADIR}/${TARGET_SYS}_config_site.d"
-
diff --git a/meta/site/common-linux b/meta/site/common-linux
new file mode 100644
index 0000000..e69de29
-- 
1.7.0.4

Re: [PATCH v2 1/1] siteinfo.bbclass: Port over oe.dev logic for site files

[Enrico Scholz]

Tom Rini <tom_rini-nmGgyN9QBj3QT0dZR+AlfA@public.gmane.org> writes:

> +python () {
> +    sitedata = set(siteinfo_data(d))
> +    if "endian-little" in sitedata:
> +        d.setVar("SITEINFO_ENDIANESS", "le")
> +    elif "endian-big" in sitedata:
> +        d.setVar("SITEINFO_ENDIANESS", "be")

Assigning SITEINFO_ENDIANESS in this way seems to break build of cross
packages (binutils, gcc).  E.g. for xscale (--> armv5te), these packages
will be build for armv5teb. This happens because tune-xscale.inc uses an
(unsafe)

| BASE_PACKAGE_ARCH = "${@['armv5teb', 'armv5te'][bb.data.getVar('SITEINFO_ENDIANESS', d, 1) == 'le']}"

statement. As there seems that something wents wrong when setting
SITEINFO_ENDIANESS in an anonymous python function, 'armv5teb' will be
assumed.

Using a more safe construct like

| BASE_PACKAGE_ARCH = "${@{'le' : 'armv5te', 'be' : 'armv5teb'}[ \
|                        bb.data.getVar('SITEINFO_ENDIANESS', d, 1)]}"

reveals that SITEINFO_ENDIANESS is undefined when expression is evaluated.

I am using bitbake master (5c8eeefc79455f058dda8f04cf4c12dc5418e00f).



Enrico

Re: [PATCH v2 1/1] siteinfo.bbclass: Port over oe.dev logic for site files

[Khem Raj]

On (25/07/11 11:57), Enrico Scholz wrote:
> Tom Rini <tom_rini-nmGgyN9QBj3QT0dZR+AlfA@public.gmane.org> writes:
> 
> > +python () {
> > +    sitedata = set(siteinfo_data(d))
> > +    if "endian-little" in sitedata:
> > +        d.setVar("SITEINFO_ENDIANESS", "le")
> > +    elif "endian-big" in sitedata:
> > +        d.setVar("SITEINFO_ENDIANESS", "be")
> 
> Assigning SITEINFO_ENDIANESS in this way seems to break build of cross
> packages (binutils, gcc).  E.g. for xscale (--> armv5te), these packages
> will be build for armv5teb. This happens because tune-xscale.inc uses an
> (unsafe)
> 
> | BASE_PACKAGE_ARCH = "${@['armv5teb', 'armv5te'][bb.data.getVar('SITEINFO_ENDIANESS', d, 1) == 'le']}"
> 
> statement. As there seems that something wents wrong when setting
> SITEINFO_ENDIANESS in an anonymous python function, 'armv5teb' will be
> assumed.

since siteinfo determines endianness based on HOST_OS this will always
be wrong for cross packages to use endianness information since it will
give back the endianness of the host system but we want the endianness
of the target system. This particularly is a problem for builds where
host != target since we are trying to use this information to construct
PACKAGE_ARCH for the cross recipes which is target dependent.

So either we need to break this link in cross.bbclass and set
BASE_PACKAGE_ARCH and PACKAGE_ARCH appropriately there.

I hope the ongoing tune file overhaul addresses this issue.

> 
> Using a more safe construct like
> 
> | BASE_PACKAGE_ARCH = "${@{'le' : 'armv5te', 'be' : 'armv5teb'}[ \
> |                        bb.data.getVar('SITEINFO_ENDIANESS', d, 1)]}"
> 
> reveals that SITEINFO_ENDIANESS is undefined when expression is evaluated.
> 
> I am using bitbake master (5c8eeefc79455f058dda8f04cf4c12dc5418e00f).
> 
> 
> 
> Enrico
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
-Khem

Re: [PATCH v2 0/1] Resync siteinfo.bbclass with oe.dev

[Saul Wold]

On 07/20/2011 01:50 PM, Tom Rini wrote:
> Hey all,
>
> This is the second of two pull requests.  This doesn't depend on the first
> request, it's just splitting changes up.  There's three changes here.  First,
> site/ix86-common was setting the wrong value for unsigned int.  Second,
> change perl to use SITEINFO variables rather than functions.
> We're down to one change now to re-sync siteinfo.bbclass with oe-core
> and oe.dev.  This re-works the functions a bit inside siteinfo.bbclass
> to be easier to add new platforms (and hopefully multilibs) to.
>
> My next step is to work on merging and verifying as much as possible
> the site files in oe-core and oe.dev.
>
> v2: Rework based on Richard's comments to not be a generator and just
> return a string list again.
>
> The following changes since commit 0d03800caffc2341b74190d38f7a372223c89f4c:
>    Mei Lei (1):
>          lsbsetup_1.0.bb: Change the hardcoded /usr/lib to support multilib
>
> are available in the git repository at:
>
>    git://git.openembedded.org/openembedded-core-contrib trini/update-siteinfo-bbclass-v2
>    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/update-siteinfo-bbclass-v2
>
> Tom Rini (1):
>    siteinfo.bbclass: Port over oe.dev logic for site files
>
>   meta/classes/siteinfo.bbclass |  203 ++++++++++++++++++++---------------------
>   1 files changed, 98 insertions(+), 105 deletions(-)
>   create mode 100644 meta/site/common-linux
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Merged into Oe-Core

Thanks
	Sau!

Re: [PATCH v2 0/1] Resync siteinfo.bbclass with oe.dev

[Koen Kooi]

Op 20 jul. 2011, om 22:50 heeft Tom Rini het volgende geschreven:

> Hey all,
> 
> This is the second of two pull requests.  This doesn't depend on the first
> request, it's just splitting changes up.  There's three changes here.  First,
> site/ix86-common was setting the wrong value for unsigned int.  Second,
> change perl to use SITEINFO variables rather than functions.  
> We're down to one change now to re-sync siteinfo.bbclass with oe-core
> and oe.dev.  This re-works the functions a bit inside siteinfo.bbclass
> to be easier to add new platforms (and hopefully multilibs) to.
> 
> My next step is to work on merging and verifying as much as possible
> the site files in oe-core and oe.dev.
> 
> v2: Rework based on Richard's comments to not be a generator and just
> return a string list again.

Not that it has been merged, can the siteinfo from meta-oe get deleted?

Re: [PATCH v2 0/1] Resync siteinfo.bbclass with oe.dev

[Tom Rini]

On 07/22/2011 08:35 AM, Koen Kooi wrote:
> 
> Op 20 jul. 2011, om 22:50 heeft Tom Rini het volgende geschreven:
> 
>> Hey all,
>>
>> This is the second of two pull requests.  This doesn't depend on the first
>> request, it's just splitting changes up.  There's three changes here.  First,
>> site/ix86-common was setting the wrong value for unsigned int.  Second,
>> change perl to use SITEINFO variables rather than functions.  
>> We're down to one change now to re-sync siteinfo.bbclass with oe-core
>> and oe.dev.  This re-works the functions a bit inside siteinfo.bbclass
>> to be easier to add new platforms (and hopefully multilibs) to.
>>
>> My next step is to work on merging and verifying as much as possible
>> the site files in oe-core and oe.dev.
>>
>> v2: Rework based on Richard's comments to not be a generator and just
>> return a string list again.
> 
> Not that it has been merged, can the siteinfo from meta-oe get deleted?

The class? yes.  It wasn't being used anyhow :)

-- 
Tom Rini
Mentor Graphics Corporation