Upstream Yocto Bringing in GCC 10

Upstream Yocto Bringing in GCC 10

[Andrew Geissler]

Ready or not, this weeks upstream update brought in GCC 10 (along with the normal slew of new packages).

The only upstream change I found we needed was a backport of a nlohmann-json GCC 10 fix[1]

For OpenBMC, we had a few minor changes spread across the repos[2]

Would appreciate if we could get our changes in quickly (before more upstream changes break other things).

With the above changes I verified we could build witherspoon and it booted up in QEMU.

If you’d like to pull the changes in for your own testing:
https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/32482/

Thanks!
Andrew

[1]: https://github.com/openembedded/meta-openembedded/pull/232
[2]: https://gerrit.openbmc-project.xyz/q/topic:%22upstream%22+(status:open)

Re: Upstream Yocto Bringing in GCC 10

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

On Sun, May 17, 2020 at 12:19:39PM -0500, Andrew Geissler wrote:
> Ready or not, this weeks upstream update brought in GCC 10 (along with the normal slew of new packages).

Alright!  The great thing about GCC 10.x is that it brings in support
for most of C++20, including co-routines.  Looking forward to playing
around with it.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Upstream Yocto Bringing in GCC 10

[Andrew Geissler]

> On May 17, 2020, at 7:08 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
> 
> On Sun, May 17, 2020 at 12:19:39PM -0500, Andrew Geissler wrote:
>> Ready or not, this weeks upstream update brought in GCC 10 (along with the normal slew of new packages).
> 
> Alright!  The great thing about GCC 10.x is that it brings in support
> for most of C++20, including co-routines.  Looking forward to playing
> around with it.

It took a few more patches but GCC 10 is now in openbmc master.
Our unit test CI framework has also been updated to run with GCC 10.

We also migrated from clang-8 to clang-10 in CI so you may want
to take a look at a recommended clang format update in this commit:
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/32696 

You may see some new CI fails with your existing code. It could be
the clang format noted above or some code that is not exercised
as a part of our normal machine builds. If you hit an issue, it’s
probably been fixed somewhere in our code base already so
browse over this commit list for your issue:
https://gerrit.openbmc-project.xyz/q/topic:%22upstream%22+(status:open%20OR%20status:merged) 

There were a lot of include related issues where you just need
to include the needed header files.

Andrew

> 
> -- 
> Patrick Williams

Re: Upstream Yocto Bringing in GCC 10

[Adrian Ambrożewicz]

W dniu 5/22/2020 o 02:36, Andrew Geissler pisze:
> 
> 
>> On May 17, 2020, at 7:08 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
>>
>> On Sun, May 17, 2020 at 12:19:39PM -0500, Andrew Geissler wrote:
>>> Ready or not, this weeks upstream update brought in GCC 10 (along with the normal slew of new packages).
>>
>> Alright!  The great thing about GCC 10.x is that it brings in support
>> for most of C++20, including co-routines.  Looking forward to playing
>> around with it.
Is it allowed in OpenBMC to base the functionality on experimental 
implementations?

> 
> It took a few more patches but GCC 10 is now in openbmc master.
> Our unit test CI framework has also been updated to run with GCC 10.
> 
> We also migrated from clang-8 to clang-10 in CI so you may want
> to take a look at a recommended clang format update in this commit:
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/32696
> 
> You may see some new CI fails with your existing code. It could be
> the clang format noted above or some code that is not exercised
> as a part of our normal machine builds. If you hit an issue, it’s
> probably been fixed somewhere in our code base already so
> browse over this commit list for your issue:
> https://gerrit.openbmc-project.xyz/q/topic:%22upstream%22+(status:open%20OR%20status:merged)
> 
> There were a lot of include related issues where you just need
> to include the needed header files.
> 
> Andrew
> 
>>
>> -- 
>> Patrick Williams
> 
Great progress :)

Regards,
Adrian

Re: Upstream Yocto Bringing in GCC 10

[Brad Bishop]

On Mon, 2020-05-25 at 14:35 +0200, Adrian Ambrożewicz wrote:
> 
> Is it allowed in OpenBMC to base the functionality on experimental 
> implementations?

I would ask the question slightly differently - will other people use my
code if I base it on experimental implementations?  At this point in
OpenBMC I think the answer is yes.  I can say it isn't something that
would stop me from using it.

thx - brad

Re: Upstream Yocto Bringing in GCC 10

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

On Mon, May 25, 2020 at 02:35:32PM +0200, Adrian Ambrożewicz wrote:
>> On May 17, 2020, at 7:08 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
>> Alright!  The great thing about GCC 10.x is that it brings in support
>> for most of C++20, including co-routines.  Looking forward to playing
>> around with it.
> Is it allowed in OpenBMC to base the functionality on experimental 
> implementations?

No disagreement with how Brad responded to this.  In the past we've been
pretty prompt at moving up to the new C++ standards.

I am curious what you meant by "experimental implementations" here
though.  Usually the C++ standards committee has put things in the
'std::experimental' namespace when they are so and the normal 'std' is
non-experimental.  This means code using 'std' APIs should continue to
work going forward, but code using 'std::experimental' might not.

The specific example I mentioned here of coroutines is out of
std::experimental as of C++20.  The compiler writers have been slow to
get it implemented because it is a complicated feature.  So, I guess
you could consider the fresh implementation at the compiler level as
"experimental" but the language / library features themselves are not?

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Upstream Yocto Bringing in GCC 10

[Adrian Ambrożewicz]

W dniu 5/26/2020 o 17:57, Patrick Williams pisze:
> On Mon, May 25, 2020 at 02:35:32PM +0200, Adrian Ambrożewicz wrote:
>>> On May 17, 2020, at 7:08 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
>>> Alright!  The great thing about GCC 10.x is that it brings in support
>>> for most of C++20, including co-routines.  Looking forward to playing
>>> around with it.
>> Is it allowed in OpenBMC to base the functionality on experimental
>> implementations?
> 
> No disagreement with how Brad responded to this.  In the past we've been
> pretty prompt at moving up to the new C++ standards.
> 
> I am curious what you meant by "experimental implementations" here
> though.  Usually the C++ standards committee has put things in the
> 'std::experimental' namespace when they are so and the normal 'std' is
> non-experimental.  This means code using 'std' APIs should continue to
> work going forward, but code using 'std::experimental' might not.
> 
> The specific example I mentioned here of coroutines is out of
> std::experimental as of C++20.  The compiler writers have been slow to
> get it implemented because it is a complicated feature.  So, I guess
> you could consider the fresh implementation at the compiler level as
> "experimental" but the language / library features themselves are not?
> 
Sure, we can distinguish 'experimental' part in two parts:
- APIs (not yet standarized),
- implementations (marked by compiler development team as experimental).

I've meant the latter. In other words - is it good to be early adopter 
of some cool new features, not yet widely tested in the field. Like 
you've said - coroutines are quite complicated feature and trusting 
early implementations might come with a risk.

I can imagine some companies or communities might choose to be careful 
in that matter. I was just wondering if there is some 'BKM' which states 
'experimental (unstable?) implementations are prohibited from use until 
marked by software vendor as stable'. Maybe that's my problem - I could 
be confusing 'experimental' with 'unstable' after all:)

Regards,
Adrian

RE: Upstream Yocto Bringing in GCC 10

[Khetan, Sharad]

-----Original Message-----
From: openbmc <openbmc-bounces+sharad.khetan=intel.com@lists.ozlabs.org> On Behalf Of Adrian Ambrozewicz
Sent: Tuesday, June 02, 2020 1:19 AM
To: Patrick Williams <patrick@stwcx.xyz>
Cc: OpenBMC Maillist <openbmc@lists.ozlabs.org>
Subject: Re: Upstream Yocto Bringing in GCC 10



W dniu 5/26/2020 o 17:57, Patrick Williams pisze:
> On Mon, May 25, 2020 at 02:35:32PM +0200, Adrian Ambrożewicz wrote:
>>> On May 17, 2020, at 7:08 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
>>> Alright!  The great thing about GCC 10.x is that it brings in 
>>> support for most of C++20, including co-routines.  Looking forward 
>>> to playing around with it.
>> Is it allowed in OpenBMC to base the functionality on experimental 
>> implementations?
> 
> No disagreement with how Brad responded to this.  In the past we've 
> been pretty prompt at moving up to the new C++ standards.
> 
> I am curious what you meant by "experimental implementations" here 
> though.  Usually the C++ standards committee has put things in the 
> 'std::experimental' namespace when they are so and the normal 'std' is 
> non-experimental.  This means code using 'std' APIs should continue to 
> work going forward, but code using 'std::experimental' might not.
> 
> The specific example I mentioned here of coroutines is out of 
> std::experimental as of C++20.  The compiler writers have been slow to 
> get it implemented because it is a complicated feature.  So, I guess 
> you could consider the fresh implementation at the compiler level as 
> "experimental" but the language / library features themselves are not?
> 
>Sure, we can distinguish 'experimental' part in two parts:
- APIs (not yet standarized),
- implementations (marked by compiler development team as experimental).

>I've meant the latter. In other words - is it good to be early adopter of some cool new features, not yet widely tested in the field. Like you've said - coroutines are quite complicated feature and trusting early implementations might come with a risk.

>I can imagine some companies or communities might choose to be careful in that matter. I was just wondering if there is some 'BKM' which states 'experimental (unstable?) implementations are prohibited from use until marked by software vendor as stable'. Maybe that's my problem - I could be confusing 'experimental' with 'unstable' after all:)

If experimental means potentially unstable, I would say we avoid such implementation in the OpenBMC. We need to keep OpenBMC stable and such new language / compiler features may be pretty gnarly to debug by the users.
 
Thanks,
-Sharad

Re: Upstream Yocto Bringing in GCC 10

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 2193 bytes --]

On Tue, Jun 02, 2020 at 06:32:34PM +0000, Khetan, Sharad wrote:
> From: openbmc <openbmc-bounces+sharad.khetan=intel.com@lists.ozlabs.org> On Behalf Of Adrian Ambrozewicz
> Sent: Tuesday, June 02, 2020 1:19 AM
> W dniu 5/26/2020 o 17:57, Patrick Williams pisze:

> >I can imagine some companies or communities might choose to be careful in that matter. I was just wondering if there is some 'BKM' which states 'experimental (unstable?) implementations are prohibited from use until marked by software vendor as stable'. Maybe that's my problem - I could be confusing 'experimental' with 'unstable' after all:)
> 
> If experimental means potentially unstable, I would say we avoid such implementation in the OpenBMC. We need to keep OpenBMC stable and such new language / compiler features may be pretty gnarly to debug by the users.

Just a few thoughts:

1. We've not historically had this distinction / restriction.  We've
   tended to use the latest C++ standard soon after it is available in our
   Yocto compiler toolchain and even used std::experimental pieces.
    * Yocto supported GCC7 (C++17 support) in April 2018 and we enabled
      it in October 2018.
    * We used mapbox::variant with C++14, which was a C++14
    * implementation of the proposed C++17 API, and then used std::variant
      when it was available.
    * We still have code referring to std::experimental::filesystem
      which has almost identical API as std::filesystem.

2. "Vendors" don't really give us an indication of stability, so we'd
   need to define one.  Today, GCC's C++20 status is a simple "no
   support" (red), "partial support" (yellow), "done" (green).  Most of
   the C++20 support is in GCC-10 which we are now using.
    * https://gcc.gnu.org/projects/cxx-status.html

3. I suspect most of our locally written code is "less stable" than
   anything coming out of GCC simply by nature of how many users and tests
   they have.  Most of the C++ features accepted by the standard
   committees have an implementation in at least 1 of the big 3
   compilers (MS-VS, GCC, Clang) and so there is code around to use for
   test suites.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Upstream Yocto Bringing in GCC 10

[Richard Hanley]

[-- Attachment #1: Type: text/plain, Size: 3063 bytes --]

I found this thread on the gcc mailing list associated with the patches to
support coroutines
https://gcc.gnu.org/legacy-ml/gcc-patches/2020-01/msg01096.html

It sounds like the main parts that are experimental are:
1) There is some debate in the standards bodies about how to handle
allocation and how to disable exceptions.
2) There isn't an agreed upon way to handle ABI differences for calling
coroutines built by another compiler

I don't expect either of those issues to be particularly relevant to
OpenBMC, which makes me more amenable to playing around with it at this
stage.

Does anyone have an idea of what a good test project would be? Something of
modest size that we can look for any hidden dragons that still need taming.

- Richard

On Tue, Jun 2, 2020 at 1:11 PM Patrick Williams <patrick@stwcx.xyz> wrote:

> On Tue, Jun 02, 2020 at 06:32:34PM +0000, Khetan, Sharad wrote:
> > From: openbmc <openbmc-bounces+sharad.khetan=intel.com@lists.ozlabs.org>
> On Behalf Of Adrian Ambrozewicz
> > Sent: Tuesday, June 02, 2020 1:19 AM
> > W dniu 5/26/2020 o 17:57, Patrick Williams pisze:
>
> > >I can imagine some companies or communities might choose to be careful
> in that matter. I was just wondering if there is some 'BKM' which states
> 'experimental (unstable?) implementations are prohibited from use until
> marked by software vendor as stable'. Maybe that's my problem - I could be
> confusing 'experimental' with 'unstable' after all:)
> >
> > If experimental means potentially unstable, I would say we avoid such
> implementation in the OpenBMC. We need to keep OpenBMC stable and such new
> language / compiler features may be pretty gnarly to debug by the users.
>
> Just a few thoughts:
>
> 1. We've not historically had this distinction / restriction.  We've
>    tended to use the latest C++ standard soon after it is available in our
>    Yocto compiler toolchain and even used std::experimental pieces.
>     * Yocto supported GCC7 (C++17 support) in April 2018 and we enabled
>       it in October 2018.
>     * We used mapbox::variant with C++14, which was a C++14
>     * implementation of the proposed C++17 API, and then used std::variant
>       when it was available.
>     * We still have code referring to std::experimental::filesystem
>       which has almost identical API as std::filesystem.
>
> 2. "Vendors" don't really give us an indication of stability, so we'd
>    need to define one.  Today, GCC's C++20 status is a simple "no
>    support" (red), "partial support" (yellow), "done" (green).  Most of
>    the C++20 support is in GCC-10 which we are now using.
>     * https://gcc.gnu.org/projects/cxx-status.html
>
> 3. I suspect most of our locally written code is "less stable" than
>    anything coming out of GCC simply by nature of how many users and tests
>    they have.  Most of the C++ features accepted by the standard
>    committees have an implementation in at least 1 of the big 3
>    compilers (MS-VS, GCC, Clang) and so there is code around to use for
>    test suites.
>
> --
> Patrick Williams
>

[-- Attachment #2: Type: text/html, Size: 3992 bytes --]

Re: Upstream Yocto Bringing in GCC 10

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

On Tue, Jun 02, 2020 at 03:12:32PM -0700, Richard Hanley wrote:
> Does anyone have an idea of what a good test project would be? Something of
> modest size that we can look for any hidden dragons that still need taming.

I'm starting work on adding coroutine support to sdbusplus as an
alternative to Boost::ASIO.  I think it would be also interesting to
look at sdeventplus for adding coroutine support and hopefully some
convergance between the two of them to facilitate coroutine reuse
(especially in the top-level scheduling portions).

My main motivations are:
    - Simplifying / flattening the callback lambdas associated with ASIO
      by using coroutines instead.

    - Use sdbusplus-coroutines as the basis for generating client
      bindings.  Since the project is moving towards async applications,
      there isn't much value in investing in sync client bindings.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Upstream Yocto Bringing in GCC 10

[William Kennington]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

Yeah, I wrote some client binding code with async support a while back but
never wrote the generators for it. I can revive the lightweight futures and
sdeventplus changes that would make them more convenient to use for this
purpose.

https://gerrit.openbmc-project.xyz/q/topic:%22sdbus-client-types%22+(status:open%20OR%20status:merged)
https://gerrit.openbmc-project.xyz/q/topic:%22sdevent-async%22+(status:open%20OR%20status:merged)

On Tue, Jun 2, 2020 at 3:55 PM Patrick Williams <patrick@stwcx.xyz> wrote:

> On Tue, Jun 02, 2020 at 03:12:32PM -0700, Richard Hanley wrote:
> > Does anyone have an idea of what a good test project would be? Something
> of
> > modest size that we can look for any hidden dragons that still need
> taming.
>
> I'm starting work on adding coroutine support to sdbusplus as an
> alternative to Boost::ASIO.  I think it would be also interesting to
> look at sdeventplus for adding coroutine support and hopefully some
> convergance between the two of them to facilitate coroutine reuse
> (especially in the top-level scheduling portions).
>
> My main motivations are:
>     - Simplifying / flattening the callback lambdas associated with ASIO
>       by using coroutines instead.
>
>     - Use sdbusplus-coroutines as the basis for generating client
>       bindings.  Since the project is moving towards async applications,
>       there isn't much value in investing in sync client bindings.
>
> --
> Patrick Williams
>

[-- Attachment #2: Type: text/html, Size: 2103 bytes --]