[PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Hongxu Jia]

Test Steps:

vim local.conf
...
MACHINE ?= "qemux86-64"
WARN_QA += "buildpaths"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "core2-32"
...

bitbake -k world

It fixed buidpath QA warning of 142 recipes (before 236, after 94) 
These newly added options does not work on native.(Which TARGET_CFLAGS
have been overrided in native.bbclass)

//Hongxu

The following changes since commit 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c:

  cmake: update to 3.4.2 (2016-01-26 22:49:40 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib hongxu/fix-buildpath
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=hongxu/fix-buildpath

Hongxu Jia (4):
  base/bbclass: use target path as compile dir in debugging info
  base/bbclass: disallow appending the compile options in debugging info
  base/bbclass: use target path as include dir in debugging info
  base/bbclass: fix missing to replace build dir in debugging info

 meta/classes/base.bbclass | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

-- 
1.9.1

[PATCH 1/4] base/bbclass: use target path as compile dir in debugging info

[Hongxu Jia]

In debugging information, it uses target paths rather than
build ones as compile dir.
...
 -fdebug-prefix-map=old=new
   When compiling files in directory old, record debugging
   information describing them as in new instead.
...

Compile without '-fdebug-prefix-map':
objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
...
| <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
|    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU C99 5.3.0 -m32
-march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack-boundary=4 -g -O2
-std=gnu99 -fgnu89-inline -feliminate-unused-debug-types -fmerge-all-constants
-frounding-math -ftls-model=initial-exec
|    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
|    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x60): /buildarea/raid0/hjia/
build-20160119-yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/git/csu
...

Compile with '-fdebug-prefix-map':
objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
...
| <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
|    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU C99 5.3.0 -m32
-march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack-boundary=4 -g -O2
-std=gnu99 -fgnu89-inline -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160119-
yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/git=/usr/src/glibc
-feliminate-unused-debug-types -fmerge-all-constants -frounding-math -ftls-model=initial-exec
|    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
|    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x60): /usr/src/glibc/csu
...

[YOCTO #7058]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes/base.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 5fc9271..3159587 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -85,6 +85,16 @@ FILESPATH = "${@base_set_filespath(["${FILE_DIRNAME}/${BP}", "${FILE_DIRNAME}/${
 # in the context of the location its used (:=)
 THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}"
 
+def debug_prefix_map(d):
+    opts = ""
+    if "-g" in (d.getVar("SELECTED_OPTIMIZATION", True) or "").split():
+        buildsrc = d.getVar("S", True)
+        targetsrc = "/usr/src/%s" % d.getVar("BPN", True)
+        opts += " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
+    return opts
+
+TARGET_CFLAGS += "${@debug_prefix_map(d)}"
+
 def extra_path_elements(d):
     path = ""
     elements = (d.getVar('EXTRANATIVEPATH', True) or "").split()
-- 
1.9.1

[PATCH 2/4] base/bbclass: disallow appending the compile options in debugging info

[Hongxu Jia]

In order to drop TMPDIR in debugging info, add option '-gno-record-gcc-switches'
to disallow appending the compile options to the DW_AT_producer attribute in
DWARF debugging information.

Compile with '-fdebug-prefix-map':
objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
...
| <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
|    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU C99 5.3.0 -m32
-march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack-boundary=4 -g -O2
-std=gnu99 -fgnu89-inline -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160119-
yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/git=/usr/src/glibc
-feliminate-unused-debug-types -fmerge-all-constants -frounding-math -ftls-model=initial-exec
|    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
|    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x60): /usr/src/glibc/csu
...

Compile with '-gno-record-gcc-switches' and '-fdebug-prefix-map':
objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
...
|    <5f>   DW_AT_producer    : (indirect string, offset: 0x1b): GNU C99 5.3.0
|    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
|    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x6e): /usr/src/glibc/csu
...

[YOCTO #7058]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes/base.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 3159587..e05552c 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -91,6 +91,7 @@ def debug_prefix_map(d):
         buildsrc = d.getVar("S", True)
         targetsrc = "/usr/src/%s" % d.getVar("BPN", True)
         opts += " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
+        opts += " -gno-record-gcc-switches"
     return opts
 
 TARGET_CFLAGS += "${@debug_prefix_map(d)}"
-- 
1.9.1

Re: [PATCH 2/4] base/bbclass: disallow appending the compile options in debugging info

[Khem Raj]

On Wed, Jan 27, 2016 at 3:45 AM, Hongxu Jia <hongxu.jia@windriver.com> wrote:
> In order to drop TMPDIR in debugging info, add option '-gno-record-gcc-switches'
> to disallow appending the compile options to the DW_AT_producer attribute in
> DWARF debugging information.

this is vital information when debugging field issues. What is the
purpose of this ?

>
> Compile with '-fdebug-prefix-map':
> objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
> ...
> | <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
> |    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU C99 5.3.0 -m32
> -march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack-boundary=4 -g -O2
> -std=gnu99 -fgnu89-inline -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160119-
> yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/git=/usr/src/glibc
> -feliminate-unused-debug-types -fmerge-all-constants -frounding-math -ftls-model=initial-exec
> |    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
> |    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x60): /usr/src/glibc/csu
> ...
>
> Compile with '-gno-record-gcc-switches' and '-fdebug-prefix-map':
> objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
> ...
> |    <5f>   DW_AT_producer    : (indirect string, offset: 0x1b): GNU C99 5.3.0
> |    <64>   DW_AT_name        : (indirect string, offset: 0xb9): init.c
> |    <68>   DW_AT_comp_dir    : (indirect string, offset: 0x6e): /usr/src/glibc/csu
> ...
>
> [YOCTO #7058]
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  meta/classes/base.bbclass | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 3159587..e05552c 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -91,6 +91,7 @@ def debug_prefix_map(d):
>          buildsrc = d.getVar("S", True)
>          targetsrc = "/usr/src/%s" % d.getVar("BPN", True)
>          opts += " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
> +        opts += " -gno-record-gcc-switches"
>      return opts
>
>  TARGET_CFLAGS += "${@debug_prefix_map(d)}"
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[PATCH 3/4] base/bbclass: use target path as include dir in debugging info

[Hongxu Jia]

In debugging information, it uses target paths rather than
build ones as include dir.

Compile without this fix:
objdump -g git/test.o
...
 The Directory Table (offset 0x1b):
|  1     /buildarea/raid0/hjia/build-20160119-yocto-buildpath/tmp/sysroots/x86_64-linux/usr/lib/
i686-pokymllib32-linux.lib32-gcc-cross-initial-i686/gcc/i686-pokymllib32-linux/5.3.0/include
|  2     /buildarea/raid0/hjia/build-20160119-yocto-buildpath/tmp/sysroots/lib32-qemux86-64/usr/include/bits
|  3     /buildarea/raid0/hjia/build-20160119-yocto-buildpath/tmp/sysroots/lib32-qemux86-64/usr/include
...

Compile with this fix:
objdump -g git/test.o
...
 The Directory Table (offset 0x1b):
|  1     /usr/lib/i686-pokymllib32-linux.lib32-gcc-cross-initial-i686/gcc/i686-pokymllib32-linux/
5.3.0/include
|  2     /usr/include/bits
|  3     /usr/include
...

[YOCTO #7058]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes/base.bbclass | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index e05552c..5c95cba 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -91,6 +91,11 @@ def debug_prefix_map(d):
         buildsrc = d.getVar("S", True)
         targetsrc = "/usr/src/%s" % d.getVar("BPN", True)
         opts += " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
+
+        for var in ["STAGING_DIR_NATIVE", "STAGING_DIR_HOST"]:
+            buildinc = d.getVar(var, True)
+            opts += " -fdebug-prefix-map=%s=" % (buildinc)
+
         opts += " -gno-record-gcc-switches"
     return opts
 
-- 
1.9.1

[PATCH 4/4] base/bbclass: fix missing to replace build dir in debugging info

[Hongxu Jia]

Previously, it missed to replace build dir(${B}).

Compile without this fix:
objdump -g packages-split/lib32-glibc-staticdev/usr/lib/librpcsvc.a
...
 The Directory Table (offset 0x1b):
|  1     /buildarea/raid0/hjia/build-20160119-yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/
lib32-glibc/2.22-r0/build-i686-pokymllib32-linux/sunrpc
|  15    /buildarea/raid0/hjia/build-20160119-yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/
lib32-glibc/2.22-r0/build-i686-pokymllib32-linux/sunrpc/rpcsvc
...

Compile with this fix:
objdump -g packages-split/lib32-glibc-staticdev/usr/lib/librpcsvc.a
...
 The Directory Table (offset 0x1b):
  1     /usr/src/glibc/sunrpc
  15    /usr/src/glibc/sunrpc/rpcsvc
...

[YOCTO #7058]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes/base.bbclass | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 5c95cba..afa7dd8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -88,9 +88,12 @@ THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}"
 def debug_prefix_map(d):
     opts = ""
     if "-g" in (d.getVar("SELECTED_OPTIMIZATION", True) or "").split():
-        buildsrc = d.getVar("S", True)
         targetsrc = "/usr/src/%s" % d.getVar("BPN", True)
-        opts += " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
+        for var in ["S", "B"]:
+            buildsrc = d.getVar(var, True)
+            opt = " -fdebug-prefix-map=%s=%s" % (buildsrc, targetsrc)
+            if opt not in opts:
+                opts += opt
 
         for var in ["STAGING_DIR_NATIVE", "STAGING_DIR_HOST"]:
             buildinc = d.getVar(var, True)
-- 
1.9.1

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Richard Purdie]

I like the idea of this a lot, I think it makes sense however why not
just:

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e80ee18..284f7fb 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
 ##################################################################
 # Optimization flags.
 ##################################################################
-DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
+DEBUG_FLAGS ?= "-g \
+    -feliminate-unused-debug-types \
+    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
+    -fdebug-prefix-map=${S}=/usr/src/${BPN} 
+    -gno-record-gcc-switches \
+    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
+    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
+"
 # Disabled until the option works properly -feliminate-dwarf2-dups
 FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
 DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"

?

Cheers,

Richard

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1429 bytes --]


> On Jan 27, 2016, at 5:16 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> I like the idea of this a lot, I think it makes sense however why not
> just:
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index e80ee18..284f7fb 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install = "${PARALLEL_MAKEINST} "
> ##################################################################
> # Optimization flags.
> ##################################################################
> -DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
> +DEBUG_FLAGS ?= "-g \
> +    -feliminate-unused-debug-types \
> +    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
> +    -fdebug-prefix-map=${S}=/usr/src/${BPN}
> +    -gno-record-gcc-switches \

can we leave the switches in ?

> +    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> +    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
> +"
> # Disabled until the option works properly -feliminate-dwarf2-dups
> FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
> DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
> 
> ?
> 
> Cheers,
> 
> Richard
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Richard Purdie]

On Wed, 2016-01-27 at 14:34 -0500, Khem Raj wrote:
> > On Jan 27, 2016, at 5:16 AM, Richard Purdie <
> > richard.purdie@linuxfoundation.org> wrote:
> > 
> > I like the idea of this a lot, I think it makes sense however why
> > not
> > just:
> > 
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index e80ee18..284f7fb 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install =
> > "${PARALLEL_MAKEINST} "
> > ##################################################################
> > # Optimization flags.
> > ##################################################################
> > -DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
> > +DEBUG_FLAGS ?= "-g \
> > +    -feliminate-unused-debug-types \
> > +    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
> > +    -fdebug-prefix-map=${S}=/usr/src/${BPN}
> > +    -gno-record-gcc-switches \
> 
> can we leave the switches in ?

Is there a way we can do some path substitutions on the switches?

I think part of the desire here is that the end resulting binaries
shouldn't depend upon the path in which they were built and leaving
these in is causing a problem there.

Cheers,

Richard

> > +    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> > +    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
> > +"
> > # Disabled until the option works properly -feliminate-dwarf2-dups
> > FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
> > DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} 
> > -pipe"
> > 
> > ?
> > 
> > Cheers,
> > 
> > Richard
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Hongxu Jia]

On 01/28/2016 06:13 AM, Richard Purdie wrote:
> On Wed, 2016-01-27 at 14:34 -0500, Khem Raj wrote:
>>> On Jan 27, 2016, at 5:16 AM, Richard Purdie <
>>> richard.purdie@linuxfoundation.org> wrote:
>>>
>>> I like the idea of this a lot, I think it makes sense however why
>>> not

My original trying was just added them here, but the variable parsing
at here is too early, the ${BPN}, ${B} and {S} was assigned with
'defaultpkgname' at this time. Which we have:
...
| -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160127-yocto-buildpath-2/
tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=
/usr/src/defaultpkgname
...

>>> just:
>>>
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index e80ee18..284f7fb 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install =
>>> "${PARALLEL_MAKEINST}"
>>> ##################################################################
>>> # Optimization flags.
>>> ##################################################################
>>> -DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
>>> +DEBUG_FLAGS ?= "-g \
>>> +    -feliminate-unused-debug-types \
>>> +    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
>>> +    -fdebug-prefix-map=${S}=/usr/src/${BPN}
>>> +    -gno-record-gcc-switches \
>> can we leave the switches in ?
> Is there a way we can do some path substitutions on the switches?

With searching the manual, I could not find a gcc option to
do the path substitution on switches in debugging information.

Maybe we could add a patch for gcc to do this. I could try in V2
if there is no side effect.

>
> I think part of the desire here is that the end resulting binaries
> shouldn't depend upon the path in which they were built and leaving
> these in is causing a problem there.

Compile with '-fdebug-prefix-map':
objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
...
| <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
|    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU C99 5.3.0 -m32
-march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack-boundary=4 -g -O2
-std=gnu99 -fgnu89-inline -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160119-
yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/git=/usr/src/glibc
-feliminate-unused-debug-types -fmerge-all-constants -frounding-math -ftls-model=initial-exec

When we use '-fdebug-prefix-map', the TMPDIR was added to switches. Although
remove the switches is to avoid buildpath QA check, the side effect is 
the other
part of compile options have also been removed.

If we need to leave the switches in, I think we could tweak the 
buildpath QA check,
which allows TMPDIR in '-fdebug-prefix-map=' without warning. I think I 
should
send a V2 to do this for discussion.

Solution1: do some path substitutions on the switches
Solution2: tweak buildpath QA check to allow TMPDIR in '-fdebug-prefix-map='

I will try them in V2.

//Hongxu

> Cheers,
>
> Richard
>
>>> +    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
>>> +    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
>>> +"
>>> # Disabled until the option works properly -feliminate-dwarf2-dups
>>> FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
>>> DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS}
>>> -pipe"
>>>
>>> ?
>>>
>>> Cheers,
>>>
>>> Richard
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Richard Purdie]

On Thu, 2016-01-28 at 10:58 +0800, Hongxu Jia wrote:
> On 01/28/2016 06:13 AM, Richard Purdie wrote:
> > On Wed, 2016-01-27 at 14:34 -0500, Khem Raj wrote:
> > > > On Jan 27, 2016, at 5:16 AM, Richard Purdie <
> > > > richard.purdie@linuxfoundation.org> wrote:
> > > > 
> > > > I like the idea of this a lot, I think it makes sense however
> > > > why
> > > > not
> 
> My original trying was just added them here, but the variable parsing
> at here is too early, the ${BPN}, ${B} and {S} was assigned with
> 'defaultpkgname' at this time. Which we have:
> ...
> > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160127-yocto
> > -buildpath-2/
> tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname
> -1.0=
> /usr/src/defaultpkgname


I think this is just from your debugging technique. By the time these
are used in recipes, they will have the correct value for these
variables. Expansion happens late, not when the line is parsed.

Can you be more specific about how you tested this and the problem you
saw?

> With searching the manual, I could not find a gcc option to
> do the path substitution on switches in debugging information.
> 
> Maybe we could add a patch for gcc to do this. I could try in V2
> if there is no side effect.

I like that idea.

> Compile with '-fdebug-prefix-map':
> objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
> ...
> > <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
> >    <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU
> > C99 5.3.0 -m32
> -march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack
> -boundary=4 -g -O2
> -std=gnu99 -fgnu89-inline -fdebug-prefix
> -map=/buildarea/raid0/hjia/build-20160119-
> yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22
> -r0/git=/usr/src/glibc
> -feliminate-unused-debug-types -fmerge-all-constants -frounding-math 
> -ftls-model=initial-exec
> 
> When we use '-fdebug-prefix-map', the TMPDIR was added to switches.
> Although
> remove the switches is to avoid buildpath QA check, the side effect
> is 
> the other
> part of compile options have also been removed.
> 
> If we need to leave the switches in, I think we could tweak the 
> buildpath QA check,
> which allows TMPDIR in '-fdebug-prefix-map=' without warning. I think
> I 
> should
> send a V2 to do this for discussion.
> 
> Solution1: do some path substitutions on the switches
> Solution2: tweak buildpath QA check to allow TMPDIR in '-fdebug
> -prefix-map='
> 
> I will try them in V2.

Thanks, that part looks good to me.

Cheers,

Richard

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Hongxu Jia]

On 01/29/2016 01:04 AM, Richard Purdie wrote:
> On Thu, 2016-01-28 at 10:58 +0800, Hongxu Jia wrote:
>> On 01/28/2016 06:13 AM, Richard Purdie wrote:
>>> On Wed, 2016-01-27 at 14:34 -0500, Khem Raj wrote:
>>>>> On Jan 27, 2016, at 5:16 AM, Richard Purdie <
>>>>> richard.purdie@linuxfoundation.org> wrote:
>>>>>
>>>>> I like the idea of this a lot, I think it makes sense however
>>>>> why
>>>>> not
>> My original trying was just added them here, but the variable parsing
>> at here is too early, the ${BPN}, ${B} and {S} was assigned with
>> 'defaultpkgname' at this time. Which we have:
>> ...
>>> -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160127-yocto
>>> -buildpath-2/
>> tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname
>> -1.0=
>> /usr/src/defaultpkgname
>
> I think this is just from your debugging technique. By the time these
> are used in recipes, they will have the correct value for these
> variables. Expansion happens late, not when the line is parsed.

For var DEBUG_FLAGS, it indeed has correct value at last, but the CFLAGS
and TARGET_CFLAGS was expanded when DEBUG_FLAGS has 'defaultpkgname'

The CFLAGS and TARGET_CFLAGS is shell variables (export CFLAGS=), it
seems shell variable is expanded only once.

>
> Can you be more specific about how you tested this and the problem you
> saw?

1. Modify bitbake.conf
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e80ee18..3ddb9e8 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install = 
"${PARALLEL_MAKEINST} "
  ##################################################################
  # Optimization flags.
  ##################################################################
-DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
+DEBUG_FLAGS ?= "-g \
+    -feliminate-unused-debug-types \
+    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
+    -fdebug-prefix-map=${S}=/usr/src/${BPN} \
+    -gno-record-gcc-switches \
+    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
+    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
+"

2. Build lib32-glibc, there are still buildpath QA Warnings
...
$ bitbake lib32-glibc
ERROR: QA Issue: File 
work/core2-32-pokymllib32-linux/lib32-glibc/2.22-r0/packages-split/lib32-glibc-dev/usr/lib/crt1.o 
in package contained reference to tmpdir
...

3. vim temp/run.do_compile
...
export 
TARGET_CFLAGS="-I/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/lib32-qemux86-64/usr/include 
-O2 -pipe -g -feliminate-unused-debug-types 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname 
-gno-record-gcc-switches 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/x86_64-linux= 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/qemux86-64= 
"

export 
CFLAGS="-I/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/lib32-qemux86-64/usr/include 
-O2 -pipe -g -feliminate-unused-debug-types 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname 
-gno-record-gcc-switches 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/x86_64-linux= 
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160128-yocto-buildpath2/tmp/sysroots/qemux86-64= 
"
...

>> With searching the manual, I could not find a gcc option to
>> do the path substitution on switches in debugging information.
>>
>> Maybe we could add a patch for gcc to do this. I could try in V2
>> if there is no side effect.
> I like that idea.

I have sent a simple modification on gcc in V2, it adds
a new option to not record  '-fdebug-prefix-map' to gcc
command line switches in debugging information.

//Hongxu

>> Compile with '-fdebug-prefix-map':
>> objdump -g packages-split/lib32-glibc-dev/usr/lib/gcrt1.o
>> ...
>>> <0><5e>: Abbrev Number: 1 (DW_TAG_compile_unit)
>>>     <5f>   DW_AT_producer    : (indirect string, offset: 0xbb): GNU
>>> C99 5.3.0 -m32
>> -march=core2 -mtune=core2 -msse3 -mfpmath=sse -mpreferred-stack
>> -boundary=4 -g -O2
>> -std=gnu99 -fgnu89-inline -fdebug-prefix
>> -map=/buildarea/raid0/hjia/build-20160119-
>> yocto-buildpath/tmp/work/core2-32-pokymllib32-linux/lib32-glibc/2.22
>> -r0/git=/usr/src/glibc
>> -feliminate-unused-debug-types -fmerge-all-constants -frounding-math
>> -ftls-model=initial-exec
>>
>> When we use '-fdebug-prefix-map', the TMPDIR was added to switches.
>> Although
>> remove the switches is to avoid buildpath QA check, the side effect
>> is
>> the other
>> part of compile options have also been removed.
>>
>> If we need to leave the switches in, I think we could tweak the
>> buildpath QA check,
>> which allows TMPDIR in '-fdebug-prefix-map=' without warning. I think
>> I
>> should
>> send a V2 to do this for discussion.
>>
>> Solution1: do some path substitutions on the switches
>> Solution2: tweak buildpath QA check to allow TMPDIR in '-fdebug
>> -prefix-map='
>>
>> I will try them in V2.
> Thanks, that part looks good to me.
>
> Cheers,
>
> Richard

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Richard Purdie]

On Fri, 2016-01-29 at 11:39 +0800, Hongxu Jia wrote:
> On 01/29/2016 01:04 AM, Richard Purdie wrote:
> > On Thu, 2016-01-28 at 10:58 +0800, Hongxu Jia wrote:
> > > On 01/28/2016 06:13 AM, Richard Purdie wrote:
> > > > On Wed, 2016-01-27 at 14:34 -0500, Khem Raj wrote:
> > > > > > On Jan 27, 2016, at 5:16 AM, Richard Purdie <
> > > > > > richard.purdie@linuxfoundation.org> wrote:
> > > > > > 
> > > > > > I like the idea of this a lot, I think it makes sense
> > > > > > however
> > > > > > why
> > > > > > not
> > > My original trying was just added them here, but the variable
> > > parsing
> > > at here is too early, the ${BPN}, ${B} and {S} was assigned with
> > > 'defaultpkgname' at this time. Which we have:
> > > ...
> > > > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160127-yocto
> > > > -buildpath-2/
> > > tmp/work/core2-64-poky-linux/defaultpkgname/1.0-r0/defaultpkgname
> > > -1.0=
> > > /usr/src/defaultpkgname
> > 
> > I think this is just from your debugging technique. By the time
> > these
> > are used in recipes, they will have the correct value for these
> > variables. Expansion happens late, not when the line is parsed.
> 
> For var DEBUG_FLAGS, it indeed has correct value at last, but the
> CFLAGS
> and TARGET_CFLAGS was expanded when DEBUG_FLAGS has 'defaultpkgname'
> 
> The CFLAGS and TARGET_CFLAGS is shell variables (export CFLAGS=), it
> seems shell variable is expanded only once.
> 
> > 
> > Can you be more specific about how you tested this and the problem
> > you
> > saw?
> 
> 1. Modify bitbake.conf
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index e80ee18..3ddb9e8 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -546,7 +546,14 @@ EXTRA_OEMAKE_prepend_task-install = 
> "${PARALLEL_MAKEINST} "
>   ##################################################################
>   # Optimization flags.
>   ##################################################################
> -DEBUG_FLAGS ?= "-g -feliminate-unused-debug-types"
> +DEBUG_FLAGS ?= "-g \
> +    -feliminate-unused-debug-types \
> +    -fdebug-prefix-map=${B}=/usr/src/${BPN} \
> +    -fdebug-prefix-map=${S}=/usr/src/${BPN} \
> +    -gno-record-gcc-switches \
> +    -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
> +    -fdebug-prefix-map=${STAGING_DIR_HOST}= \
> +"
> 
> 2. Build lib32-glibc, there are still buildpath QA Warnings

I investigated this a little. When I tried your patch I saw:

$ bitbake glibc -e | grep TARGET_CFLAGS=
export TARGET_CFLAGS="
-I/media/build1/poky/build/tmp5/sysroots/qemux86/usr/include -O2 -pipe 
-g -feliminate-unused-debug-types -fdebug-prefix
-map=/media/build1/poky/build/tmp5/work/i586-poky
-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname
-fdebug-prefix-map=/media/build1/poky/build/tmp5/work/i586-poky
-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname
-gno-record-gcc-switches -fdebug-prefix
-map=/media/build1/poky/build/tmp5/sysroots/x86_64-linux= -fdebug
-prefix-map=/media/build1/poky/build/tmp5/sysroots/qemux86= "

and then once I commented out:

SELECTED_OPTIMIZATION := "${@get_optimization(d)}"

in glibc.inc I see:

$ bitbake glibc -e | grep TARGET_CFLAGS=
export TARGET_CFLAGS="
-I/media/build1/poky/build/tmp5/sysroots/qemux86/usr/include -O2 -pipe 
-g -feliminate-unused-debug-types -fdebug-prefix
-map=/media/build1/poky/build/tmp5/work/i586-poky-linux/glibc/2.22
-r0/build-i586-poky-linux=/usr/src/glibc -fdebug-prefix
-map=/media/build1/poky/build/tmp5/work/i586-poky-linux/glibc/2.22
-r0/git=/usr/src/glibc -gno-record-gcc-switches -fdebug-prefix
-map=/media/build1/poky/build/tmp5/sysroots/x86_64-linux= -fdebug
-prefix-map=/media/build1/poky/build/tmp5/sysroots/qemux86= "

I'd much prefer to fix glibc.inc and put the entries in bitbake.conf
than further complicate the variables unnecessarily.

For example, we could do:

diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
index 17fa2d5..8e62b3a 100644
--- a/meta/recipes-core/glibc/glibc.inc
+++ b/meta/recipes-core/glibc/glibc.inc
@@ -10,25 +10,20 @@ TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TCBOOTSTRAP}"
 
 # glibc can't be built without optimization, if someone tries to compile an
 # entire image as -O0, we override it with -O2 here and give a note about it.
-def get_optimization(d):
+python () {
     selected_optimization = d.getVar("SELECTED_OPTIMIZATION", True)
-    if bb.utils.contains("SELECTED_OPTIMIZATION", "-O2", "x", "", d) == "x":
-        return selected_optimization
-    elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O", "x", "", d) == "x":
+    if bb.utils.contains("SELECTED_OPTIMIZATION", "-O", "x", "", d) == "x":
         bb.note("glibc can't be built with -O, -O -Wno-error will be used instead.")
-        return selected_optimization.replace("-O", "-O -Wno-error")
+        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
     elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O0", "x", "", d) == "x":
-        bb.note("glibc can't be built with -O0, -O2 will be used instead.")
-        return selected_optimization.replace("-O0", "-O2")
+        bb.fatal("glibc can't be built with -O0, using -O1 -Wno-error or -O1 instead.")
     elif bb.utils.contains("SELECTED_OPTIMIZATION", "-Os", "x", "", d) == "x":
         bb.note("glibc can't be built with -Os, -Os -Wno-error will be used instead.")
-        return selected_optimization.replace("-Os", "-Os -Wno-error")
+        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
     elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O1", "x", "", d) == "x":
         bb.note("glibc can't be built with -O1, -O1 -Wno-error will be used instead.")
-        return selected_optimization.replace("-O1", "-O1 -Wno-error")
-    return selected_optimization
-
-SELECTED_OPTIMIZATION := "${@get_optimization(d)}"
+        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
+}
 
 # siteconfig.bbclass runs configure which needs a working compiler
 # For the compiler to work we need a working libc yet libc isn't

which will now give an error for -O0 but I think that is reasonable.
I'd even prefer to drop the above entirely and use bitbake.conf to be
honest but I know some people insisted we should have the above.

Cheers,

Richard

Re: [PATCH 0/4] base/bbclass: use target path to replace build ones in debugging info

[Hongxu Jia]

On 01/29/2016 02:40 PM, Richard Purdie wrote:
>
> I investigated this a little. When I tried your patch I saw:
>
> $ bitbake glibc -e | grep TARGET_CFLAGS=
> export TARGET_CFLAGS="
> -I/media/build1/poky/build/tmp5/sysroots/qemux86/usr/include -O2 -pipe
> -g -feliminate-unused-debug-types -fdebug-prefix
> -map=/media/build1/poky/build/tmp5/work/i586-poky
> -linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname
> -fdebug-prefix-map=/media/build1/poky/build/tmp5/work/i586-poky
> -linux/defaultpkgname/1.0-r0/defaultpkgname-1.0=/usr/src/defaultpkgname
> -gno-record-gcc-switches -fdebug-prefix
> -map=/media/build1/poky/build/tmp5/sysroots/x86_64-linux= -fdebug
> -prefix-map=/media/build1/poky/build/tmp5/sysroots/qemux86= "
>
> and then once I commented out:
>
> SELECTED_OPTIMIZATION := "${@get_optimization(d)}"
>
> in glibc.inc I see:
>
> $ bitbake glibc -e | grep TARGET_CFLAGS=
> export TARGET_CFLAGS="
> -I/media/build1/poky/build/tmp5/sysroots/qemux86/usr/include -O2 -pipe
> -g -feliminate-unused-debug-types -fdebug-prefix
> -map=/media/build1/poky/build/tmp5/work/i586-poky-linux/glibc/2.22
> -r0/build-i586-poky-linux=/usr/src/glibc -fdebug-prefix
> -map=/media/build1/poky/build/tmp5/work/i586-poky-linux/glibc/2.22
> -r0/git=/usr/src/glibc -gno-record-gcc-switches -fdebug-prefix
> -map=/media/build1/poky/build/tmp5/sysroots/x86_64-linux= -fdebug
> -prefix-map=/media/build1/poky/build/tmp5/sysroots/qemux86= "
>
> I'd much prefer to fix glibc.inc and put the entries in bitbake.conf
> than further complicate the variables unnecessarily.
>
> For example, we could do:
>
> diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
> index 17fa2d5..8e62b3a 100644
> --- a/meta/recipes-core/glibc/glibc.inc
> +++ b/meta/recipes-core/glibc/glibc.inc
> @@ -10,25 +10,20 @@ TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TCBOOTSTRAP}"
>   
>   # glibc can't be built without optimization, if someone tries to compile an
>   # entire image as -O0, we override it with -O2 here and give a note about it.
> -def get_optimization(d):
> +python () {
>       selected_optimization = d.getVar("SELECTED_OPTIMIZATION", True)
> -    if bb.utils.contains("SELECTED_OPTIMIZATION", "-O2", "x", "", d) == "x":
> -        return selected_optimization
> -    elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O", "x", "", d) == "x":
> +    if bb.utils.contains("SELECTED_OPTIMIZATION", "-O", "x", "", d) == "x":
>           bb.note("glibc can't be built with -O, -O -Wno-error will be used instead.")
> -        return selected_optimization.replace("-O", "-O -Wno-error")
> +        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
>       elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O0", "x", "", d) == "x":
> -        bb.note("glibc can't be built with -O0, -O2 will be used instead.")
> -        return selected_optimization.replace("-O0", "-O2")
> +        bb.fatal("glibc can't be built with -O0, using -O1 -Wno-error or -O1 instead.")
>       elif bb.utils.contains("SELECTED_OPTIMIZATION", "-Os", "x", "", d) == "x":
>           bb.note("glibc can't be built with -Os, -Os -Wno-error will be used instead.")
> -        return selected_optimization.replace("-Os", "-Os -Wno-error")
> +        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
>       elif bb.utils.contains("SELECTED_OPTIMIZATION", "-O1", "x", "", d) == "x":
>           bb.note("glibc can't be built with -O1, -O1 -Wno-error will be used instead.")
> -        return selected_optimization.replace("-O1", "-O1 -Wno-error")
> -    return selected_optimization
> -
> -SELECTED_OPTIMIZATION := "${@get_optimization(d)}"
> +        d.appendVar("SELECTED_OPTIMIZATION", " -Wno-error")
> +}
>   
>   # siteconfig.bbclass runs configure which needs a working compiler
>   # For the compiler to work we need a working libc yet libc isn't
>
> which will now give an error for -O0 but I think that is reasonable.
> I'd even prefer to drop the above entirely and use bitbake.conf to be
> honest but I know some people insisted we should have the above.

Yes, you are right! They immediate expand SELECTED_OPTIMIZATION
in glibc and systemtap. I will modify both of them by 'anonymous python
function to replace' to replace in V3.

And then, I will directly put the entries in bitbake.conf.

V3 incoming

//Hongxu
> Cheers,
>
> Richard