[meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[robert.berger.oe.devel]

Hi,

I tried to add support for a host gcc-9 and suggested this patch[1]

[1] 
https://github.com/RobertBerger/meta-java/commit/898a0ae33c9102387aae2e3427a007e3935e663e

and, as per your suggestion, since it restricts the host compiler to >= 
gcc9 I added "-Wno-unknown-warning".

https://github.com/RobertBerger/meta-java/commit/3d8ae5afb639ec7abcc48f15c74b42b373863c57

... but it does not build:

https://pastebin.com/B1nxHcPH

cc1plus: error: unrecognized command line option ‘-Wno-unknown-warning’ 
[-Werror]

That's my host gcc:

pokyuser@3a469e7032d9:/workdir$ gcc --version
gcc (Ubuntu 9.2.1-17ubuntu1~16.04) 9.2.1 20191102
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

pokyuser@3a469e7032d9:/workdir$

Please advise.

Regards,

Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Richard Leitner]

Hi Robert,
sorry for the delay.

I pushed the "g0hl1n/jdk8-fix" branch to the meta-java git repository.
It is a temporary test branch and it should fix the GCC version
dependencies (as long as you have GCC >= v6).

It would be great if you could do a quick test and give me feedback ;-)

thanks & regards;rl

On Fri, Mar 20, 2020 at 06:53:17PM +0200, robert.berger.oe.devel wrote:
> Hi,
> 
> I tried to add support for a host gcc-9 and suggested this patch[1]
> 
> [1] https://github.com/RobertBerger/meta-java/commit/898a0ae33c9102387aae2e3427a007e3935e663e
> 
> and, as per your suggestion, since it restricts the host compiler to >= gcc9
> I added "-Wno-unknown-warning".
> 
> https://github.com/RobertBerger/meta-java/commit/3d8ae5afb639ec7abcc48f15c74b42b373863c57
> 
> ... but it does not build:
> 
> https://pastebin.com/B1nxHcPH
> 
> cc1plus: error: unrecognized command line option ‘-Wno-unknown-warning’
> [-Werror]
> 
> That's my host gcc:
> 
> pokyuser@3a469e7032d9:/workdir$ gcc --version
> gcc (Ubuntu 9.2.1-17ubuntu1~16.04) 9.2.1 20191102
> Copyright (C) 2019 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> 
> pokyuser@3a469e7032d9:/workdir$
> 
> Please advise.
> 
> Regards,
> 
> Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

My comments are in line.

On 29/04/2020 13:02, Richard Leitner wrote:
> Hi Robert,
> sorry for the delay.

No prob. I know what it means to be busy ;)

> 
> I pushed the "g0hl1n/jdk8-fix" branch to the meta-java git repository.
> It is a temporary test branch and it should fix the GCC version
> dependencies (as long as you have GCC >= v6).
> 
> It would be great if you could do a quick test and give me feedback ;-)

You mean that should work against dunfell?

I will give it a try.

... for the archiver issue I think I might be able to figure out in 
Python who called the function and react accordingly. Will have a look 
at this as well as time permits.

> 
> thanks & regards;rl

Regards,

Robert

> 
> On Fri, Mar 20, 2020 at 06:53:17PM +0200, robert.berger.oe.devel wrote:
>> Hi,
>>
>> I tried to add support for a host gcc-9 and suggested this patch[1]
>>
>> [1] https://github.com/RobertBerger/meta-java/commit/898a0ae33c9102387aae2e3427a007e3935e663e
>>
>> and, as per your suggestion, since it restricts the host compiler to >= gcc9
>> I added "-Wno-unknown-warning".
>>
>> https://github.com/RobertBerger/meta-java/commit/3d8ae5afb639ec7abcc48f15c74b42b373863c57
>>
>> ... but it does not build:
>>
>> https://pastebin.com/B1nxHcPH
>>
>> cc1plus: error: unrecognized command line option ‘-Wno-unknown-warning’
>> [-Werror]
>>
>> That's my host gcc:
>>
>> pokyuser@3a469e7032d9:/workdir$ gcc --version
>> gcc (Ubuntu 9.2.1-17ubuntu1~16.04) 9.2.1 20191102
>> Copyright (C) 2019 Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions.  There is NO
>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>>
>> pokyuser@3a469e7032d9:/workdir$
>>
>> Please advise.
>>
>> Regards,
>>
>> Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Richard Leitner]

On Wed, Apr 29, 2020 at 02:41:20PM +0300, robert.berger.oe.devel wrote:
> On 29/04/2020 13:02, Richard Leitner wrote:

> > 
> > I pushed the "g0hl1n/jdk8-fix" branch to the meta-java git repository.
> > It is a temporary test branch and it should fix the GCC version
> > dependencies (as long as you have GCC >= v6).
> > 
> > It would be great if you could do a quick test and give me feedback ;-)
> 
> You mean that should work against dunfell?

Yes, the branch is for dunfell. Nonetheless it should be quite easy
to backport the following commits to other releases:

commit 49b1405 ("openjdk-8: remove -Werror")
commit e3d7bce ("openjdk-8: require GCC >= v6")
commit cc2e1b2 ("openjdk-8: update to latest ga version 252")

> 
> I will give it a try.

Great! Thanks.

> 
> ... for the archiver issue I think I might be able to figure out in Python
> who called the function and react accordingly. Will have a look at this as
> well as time permits.

Ok. Thank you.

regards;rl

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

Is this supposed to work with host gcc 9?

This was the first test I did and it bombed.

I used all upstream unmodified dunfell branches plus your branch.

pokyuser@2f9b1db70cea:/workdir/build$ gcc --version
gcc (Ubuntu 9.2.1-17ubuntu1~16.04) 9.2.1 20191102
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

pokyuser@2f9b1db70cea:/workdir/build$

Build Configuration:
BB_VERSION           = "1.46.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-16.04"
TARGET_SYS           = "x86_64-resy-linux"
MACHINE              = "container-x86-64"
DISTRO               = "resy-container"
DISTRO_VERSION       = "3.1"
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
meta
meta-poky
meta-yocto-bsp       = 
"2020-04-30-dunfell-3.1:5e44568d90cb9148253098f60a6aeb32c0e5cf5f"
meta-resy            = "dunfell:572324800b1a33a30d6eba67ac343cbc67e9737a"
meta-oe
meta-python          = 
"2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056"
meta-virtualization  = 
"2020-04-30-dunfell-3.1:e319e746bffc4360229b2c926b8c5c9492ee52a7"
meta-filesystems
meta-networking      = 
"2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056"
meta-java            = 
"2020-04-30-g0hl1n/jdk8-fix_local:49b1405f5429a1470f28823d726fcb1b67d3bce3"

...

ERROR: Task 
(/workdir/sources/poky/../meta-java/recipes-core/icedtea/icedtea7-native_2.1.3.bb:do_compile) 
failed with exit code '1'

here is the error:

https://pastebin.com/x4c1Kduw

Regards,

Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

I started to have a look into the icetea-native issue again and hope I 
will be able to come up with some patch, which fixes it host-gcc dependent.

The idea is to use a hacked version of this[1] and add it to [2].

Like this I should be able to retrieve the host gcc version dynamically 
and be able to add and remove CFLAGS e.g. depending on the major version 
of the host gcc.

Regards,

Robert

[1] 
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/utils.py?h=dunfell#n377

[2] 
http://git.yoctoproject.org/cgit/cgit.cgi/meta-java/tree/classes/openjdk-build-helper.bbclass?h=g0hl1n/jdk8-fix

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Richard Leitner]

On Thu, Apr 30, 2020 at 10:27:12PM +0300, robert.berger.oe.devel wrote:
> Hi,
> 
> I started to have a look into the icetea-native issue again and hope I will
> be able to come up with some patch, which fixes it host-gcc dependent.
> 
> The idea is to use a hacked version of this[1] and add it to [2].
> 
> Like this I should be able to retrieve the host gcc version dynamically and
> be able to add and remove CFLAGS e.g. depending on the major version of the
> host gcc.

If I understand you correctly we have something like this already [3].
Used for example at [4].

Or did I got something wrong?

regards;rl

[3] https://git.yoctoproject.org/cgit/cgit.cgi/meta-java/tree/classes/openjdk-build-helper.bbclass?h=g0hl1n/jdk8-fix#n24

[4] https://git.yoctoproject.org/cgit/cgit.cgi/meta-java/tree/recipes-core/openjdk/openjdk-8-common.inc#n178

> 
> Regards,
> 
> Robert
> 
> [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/utils.py?h=dunfell#n377
> 
> [2] http://git.yoctoproject.org/cgit/cgit.cgi/meta-java/tree/classes/openjdk-build-helper.bbclass?h=g0hl1n/jdk8-fix

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi Richard,

What do you think about this fix? [1]

[1] 
https://github.com/RobertBerger/meta-java/commit/3db52ddcb7771e989cf39048a7b084d040d6b5b0

I am still building and testing, but it looks like this allows me to 
successfully build a jdk for x86 which runs in an x86 container with gcc-9.

It conditionally adds the required CLFAGS.

I will run a few more tests and see if it works as expected also for 
older gcc versions. When I tried last time I was only able to create a 
jdk with gcc-9 and whatever was on master-next which was also able to 
run without crashing. I am curious if with the new version something 
changed.

Run like in:

root@f1e0f8e7ef93:/# java -version
openjdk version "1.8.0_252-internal"
OpenJDK Runtime Environment (build 1.8.0_252-internal-ga)
OpenJDK 64-Bit Server VM (build 25.252-bga, mixed mode)

Regards,

Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

Please see my comments in-line.

On 01/05/2020 00:17, Richard Leitner wrote:
>>
>> Like this I should be able to retrieve the host gcc version dynamically and
>> be able to add and remove CFLAGS e.g. depending on the major version of the
>> host gcc.
> 
> If I understand you correctly we have something like this already [3].
> Used for example at [4].
> 
> Or did I got something wrong?

I think this is along the lines of what I want.

... but ...

Why do we need to use "custom" meta-java stuff instead of what's already 
available from OE out of the box?

HOST_COMPILER_MAJOR="${@oe.utils.get_host_compiler_version(d)[1].split(".", 
2)[0]}"
CFLAGS_append = " ${@oe.utils.conditional('HOST_COMPILER_MAJOR', '9', 
'-Wno-error=format-overflow', '', d)}"

works for me.

... and ...

It looks like your and my build environment with gcc-9 (RH) and gcc-9 
(UB) behave differently?

This is totally crazy. Not only do we have a host compiler dependency 
with meta-java, but also 9.2 vs. 9.3 and maybe RH and UB hacked 
different things there?

Also we are just turning off warnings/errors, in this case buffer 
overflows, instead of fixing them, which is not good as well. So the 
problem might also be in some library/header or so. But I guess with 
OpenJDK to fix those it's another cup of tea.

... or ...

somewhere magic host CFLAGS are being added.

... one more thing ...

Do you think it should work on one of those distros out of the box, so I 
can try to reproduce what you have on my end?

https://github.com/RobertBerger/yocto-dockerfiles/tree/master/dockerfiles

How is meta-java usually tested?

> 
> regards;rl
> 

Regards,

Robert

Re: [oe] [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

Just for completeness I tried with a

gcc (Ubuntu 9.3.0-10ubuntu2~16.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

same behavior. Compiler refuses to compile without my patch.

Just looking at the code I believe the compiler should complain unless 
you tell it otherwise.

That's what it complains about:

| In function ‘int fprintf(FILE*, const char*, ...)’,
|     inlined from ‘FormatRule* ADLParser::template_parse()’ at 
/workdir/build/container-x86-64-java/tmp/work/x86_64-linux/icedtea7-native/2.1.3-r1.0/icedtea-2.1.3/build/openjdk-boot/hotspot/src/share/vm/adlc/adlparse.cpp:3803:34:
| /usr/include/x86_64-linux-gnu/bits/stdio2.h:97:24: error: ‘%s’ 
directive argument is null [-Werror=format-overflow=]
|    97 |   return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1, __fmt,
|       |          ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|    98 |    __va_arg_pack ());
|       |    ~~~~~~~~~~~~~~~~~


If we look at the source code:

vim 
/workdir/build/container-x86-64-java/tmp/work/x86_64-linux/icedtea7-native/2.1.3-r1.0/icedtea-2.1.3/build/openjdk-boot/hotspot/src/share/vm/adlc/adlparse.cpp 
+3803

   // Debug Stuff
   if (_AD._adl_debug > 1) fprintf(stderr,"Format Rule: %s\n", desc);

At the beginnig of this method you can see:

//------------------------------template_parse-----------------------------------
FormatRule* ADLParser::template_parse(void) {
   char       *desc   = NULL;
   FormatRule *format = (new FormatRule(desc));
...

So I guess the compiler is wrong to complain, since desc could be NULL.

Regards,

Robert

Re: [oe] [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

So I guess the compiler is __NOT__ wrong to complain, since desc could 
be NULL.

Regards,

Robert

Re: [oe] [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Richard Leitner]

Hi Robert,

I'm just responding to your last message as you were completely right
with your problem.

My fix covered only openjdk-8 and not icedtea7-native. I've overlooked
that somehow ;-)

Therefore it would be great if you can provide a patch to the
mailinglist for fixing the issue.

Thanks for your patience!

regards;rl

On Fri, May 01, 2020 at 05:57:18PM +0300, Robert Berger wrote:
> So I guess the compiler is __NOT__ wrong to complain, since desc could be
> NULL.
> 
> Regards,
> 
> Robert
> 
> 

Re: [oe] [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

On 05/05/2020 08:35, Richard Leitner wrote:
> My fix covered only openjdk-8 and not icedtea7-native. I've overlooked
> that somehow ;-)
> 
> Therefore it would be great if you can provide a patch to the
> mailinglist for fixing the issue.
> 
> Thanks for your patience!

Stay tuned. I am currently busy, but I hope I will be able to send out 
my proposed patch this or next week.

I assume something similar might be needed for gcc-10 as well.

> 
> regards;rl
> 

Regards,

Robert

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Richard Leitner]

On Thu, Apr 30, 2020 at 06:43:46PM +0300, Robert Berger wrote:
> Hi,
> 
> Is this supposed to work with host gcc 9?

Yes, for me it works with "gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)"

> 
> This was the first test I did and it bombed.
> 
> I used all upstream unmodified dunfell branches plus your branch.

same did I on my Fedora box and it built and succeeded tests... That's
strange.

Have you a clean build?

regards;rl

> Regards,
> 
> Robert
> 

Re: [meta-java] host gcc-9, icetea-7 gcc-9 but how older gcc versions?

[Robert Berger]

Hi,

Please see my comments in-line.

On 01/05/2020 00:13, Richard Leitner wrote:
> On Thu, Apr 30, 2020 at 06:43:46PM +0300, Robert Berger wrote:
>> Hi,
>>
>> Is this supposed to work with host gcc 9?
> 
> Yes, for me it works with "gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)"

Hmm for gcc (Ubuntu 9.2.1-17ubuntu1~16.04) 9.2.1 20191102

it needs additional CFLAGS.

Do we really need to distinguish also Distros and/or 9.3 vs. 9.2??

Does it do any harm if you add this?

https://github.com/RobertBerger/meta-java/commit/3db52ddcb7771e989cf39048a7b084d040d6b5b0

> 
>>
>> This was the first test I did and it bombed.
>>
>> I used all upstream unmodified dunfell branches plus your branch.
> 
> same did I on my Fedora box and it built and succeeded tests... That's
> strange.
> 
> Have you a clean build?

Yes all fresh and also no SSTATE or SSTATE cache.

If you would like to reproduce the problem I can provide you with ssh 
access to my build environment to show you quickly.

The build docker container I use is this one:

https://hub.docker.com/r/reliableembeddedsystems/poky-container/tags

Tag: ubuntu-16.04-gcc-9

Also I believe it can be easily reproduced with a "dirty" build as well, 
if I remove my 2 lines and just build icedtea-native.

> 
> regards;rl
> 
>> Regards,
>>
>> Robert
>>

Regards,

Robert