moving meta-{openpower, x86, arm} content to meta-phosphor

moving meta-{openpower, x86, arm} content to meta-phosphor

[Brad Bishop]

Fellow OpenBMCers

Over time, I would like to do away with the processor arch layers e.g.

meta-openpower, meta-arm, meta-x86.

In hindsight meta-arm and meta-x86 might not have made much sense and
should have been something like meta-x86-intel and meta-x86-amd perhaps
- this is confirmed by the fact that there isn't any content in meta-
x86.

I propose the content simply go in meta-phosphor, and that we frame our
thinking of meta-phosphor and OpenBMC as a project that supports any and
all host processor architectures (as well as devices that aren't servers
at all).  This results in several positive things:

- Increased developer/maintainer awareness/cross pollination of other
usage patterns (community building).
- Differences are obvious, highlighting areas for improvement in the
project.
- Build time, cross arch incompatibilities become obvious (e.g. building
images that support both Intel and AMD processors for example).
- Improved time to understanding for newcomers - everything is one
place.
- Reduced (granted a small amount) layer configuration complexity for
end users.

I'm not aware of any benefits to factoring things out into the different
layers like we have today - if you are aware of something, please share.

Getting more detailed, how would this look?  This series is an example:

https://gerrit.openbmc-project.xyz/35759

For projects that are truly host processor specific e.g. peci or occ
support, we already have a recipes-x86 folder:

https://github.com/openbmc/meta-phosphor/tree/master/recipes-x86

I propose we allow the creation of additional folders using this
convention e.g.

- recipes-power
- recipes-x86-amd (we might want to look at renaming recipes-x86 to
recipes-x86-intel)

Or even better IMO, we co-mingle these recipes as well based on the
abstract function they provide for some of the same reasons I would like
to move to a single layer - increased awareness of what your community
peers are up to.

Please share your thoughts on the matter.

thx - brad

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Supreeth Venkatesh]

On 8/20/20 8:15 AM, Brad Bishop wrote:
> [CAUTION: External Email]
> 
> Fellow OpenBMCers
Hi Brad,
> 
> Over time, I would like to do away with the processor arch layers e.g.
> 
> meta-openpower, meta-arm, meta-x86.
> 
> In hindsight meta-arm and meta-x86 might not have made much sense and
> should have been something like meta-x86-intel and meta-x86-amd perhaps
> - this is confirmed by the fact that there isn't any content in meta-
> x86.
> 
> I propose the content simply go in meta-phosphor, and that we frame our
> thinking of meta-phosphor and OpenBMC as a project that supports any and
> all host processor architectures (as well as devices that aren't servers
> at all).  This results in several positive things:
> 
> - Increased developer/maintainer awareness/cross pollination of other
> usage patterns (community building).
> - Differences are obvious, highlighting areas for improvement in the
> project.
> - Build time, cross arch incompatibilities become obvious (e.g. building
> images that support both Intel and AMD processors for example).
> - Improved time to understanding for newcomers - everything is one
> place.
> - Reduced (granted a small amount) layer configuration complexity for
> end users.
> 
> I'm not aware of any benefits to factoring things out into the different
> layers like we have today - if you are aware of something, please share.
> 
> Getting more detailed, how would this look?  This series is an example:
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgerrit.openbmc-project.xyz%2F35759&data=02%7C01%7CSupreeth.Venkatesh%40amd.com%7Cbbd5c3a67de445e0fefe08d8450b6d82%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637335262670832408&sdata=NVk%2Bs5DTzdtp2t36ZJ20yvF7lrJiy81GjX5dzM6leRo%3D&reserved=0
> 
> For projects that are truly host processor specific e.g. peci or occ
> support, we already have a recipes-x86 folder:
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenbmc%2Fmeta-phosphor%2Ftree%2Fmaster%2Frecipes-x86&data=02%7C01%7CSupreeth.Venkatesh%40amd.com%7Cbbd5c3a67de445e0fefe08d8450b6d82%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637335262670832408&sdata=jrectSdNgbMaWl%2BgtjF8gNbyAEbgbecmQjvelruheR4%3D&reserved=0
PECI is specific to Intel host x86 Architecture.
APML is used for AMD host x86 architecture.
> 
> I propose we allow the creation of additional folders using this
> convention e.g.
> 
> - recipes-power
> - recipes-x86-amd (we might want to look at renaming recipes-x86 to
> recipes-x86-intel)
Yes. It would be apt to rename recipes-x86 to recipes-x86-intel and recipes-x86-amd because of differences noted above for now.

> 
> Or even better IMO, we co-mingle these recipes as well based on the
> abstract function they provide for some of the same reasons I would like
> to move to a single layer - increased awareness of what your community
> peers are up to.
This is interesting, I like this approach as I hope this will lead to more collaboration within the community.
+1
> 
> Please share your thoughts on the matter.
> 
> thx - brad
> 

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]

On Thu, Aug 20, 2020 at 09:15:52AM -0400, Brad Bishop wrote:
> I propose we allow the creation of additional folders using this
> convention e.g.
> 
> - recipes-power

I'd like to propose a change to the name of your processor
architecture to avoid confusion between recipes involving the power
subsystem, but I'm sure your marketing organization would have a thing
or two to say about it.  In seriousness, it might be good to continue to
use openpower in this project considering that the OpenPower Foundation
holds the ISA specs and it avoids confusion with the power subsystem.

> - recipes-x86-amd (we might want to look at renaming recipes-x86 to
> recipes-x86-intel)

I think it would be good to come up with a schema on how we represent
the machine overrides and recipe subdirectories so there isn't
inconsistency there.   Something like <arch>-<company>-<model>[1]?

I do have slight concern about there becoming an enormous number of
variable overrides, patch files, etc. as we support an increasing number
of processors, but I suppose that points to an underlying problem in our
implementation which needs refactoring.


1. What do we do about risc-v which has a dash in the architecture name?

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Ed Tanous]

On Thu, Aug 20, 2020 at 9:32 AM Patrick Williams <patrick@stwcx.xyz> wrote:
>
> On Thu, Aug 20, 2020 at 09:15:52AM -0400, Brad Bishop wrote:
> > I propose we allow the creation of additional folders using this
> > convention e.g.
> >
> > - recipes-power
>
> I'd like to propose a change to the name of your processor
> architecture to avoid confusion between recipes involving the power
> subsystem, but I'm sure your marketing organization would have a thing
> or two to say about it.  In seriousness, it might be good to continue to
> use openpower in this project considering that the OpenPower Foundation
> holds the ISA specs and it avoids confusion with the power subsystem.
>
> > - recipes-x86-amd (we might want to look at renaming recipes-x86 to
> > recipes-x86-intel)
>
> I think it would be good to come up with a schema on how we represent
> the machine overrides and recipe subdirectories so there isn't
> inconsistency there.   Something like <arch>-<company>-<model>[1]?

I think in this, recipes-power would go to recipes-power-IBM.  I think
that solves it, no?

>
> I do have slight concern about there becoming an enormous number of
> variable overrides, patch files, etc. as we support an increasing number
> of processors, but I suppose that points to an underlying problem in our
> implementation which needs refactoring.
>
>
> 1. What do we do about risc-v which has a dash in the architecture name?

Technically isn't RISC-V the same thing as power9, and we just drop
the version from it?  Or is the "5" an important part of the name?
RISC-V is definitely a case where having the company in the naming
convention is going to be important, given that (to my understanding)
the design can be picked up by anyone and modded as they see fit.

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On Fri, Aug 21, 2020 at 09:53:21AM -0700, Ed Tanous wrote:
> On Thu, Aug 20, 2020 at 9:32 AM Patrick Williams <patrick@stwcx.xyz> wrote:
> > 1. What do we do about risc-v which has a dash in the architecture name?
> 
> Technically isn't RISC-V the same thing as power9, and we just drop
> the version from it?  Or is the "5" an important part of the name?
> RISC-V is definitely a case where having the company in the naming
> convention is going to be important, given that (to my understanding)
> the design can be picked up by anyone and modded as they see fit.

risc-v is the full architecture name, not the chip model name.

Linux has chosen to put it in a 'riscv' subdirectory though, so dropping
the dash seems entirely reasonable.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[郁雷]

On Thu, Aug 20, 2020 at 9:17 PM Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
>
> Please share your thoughts on the matter.
>

The whole idea looks good and it makes the layers simpler.

One concern would be, what if we have recipes with the same name in
(previous) different layers?

E.g. in case we had:
  meta-openpower/recipes-xxx/foo1/bar.bb
  meta-x86/recipes-xxx/foo2/bar.bb
And if we move the layers into the meta-phosphor, and a build picks
the bar.bb recipe, what will happen?


-- 
BRs,
Lei YU

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Ed Tanous]

On Thu, Aug 20, 2020 at 11:23 PM 郁雷 <yulei.sh@bytedance.com> wrote:
>
> On Thu, Aug 20, 2020 at 9:17 PM Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
> >
> > Please share your thoughts on the matter.
> >
>
> The whole idea looks good and it makes the layers simpler.
>
> One concern would be, what if we have recipes with the same name in
> (previous) different layers?
>
> E.g. in case we had:
>   meta-openpower/recipes-xxx/foo1/bar.bb
>   meta-x86/recipes-xxx/foo2/bar.bb
> And if we move the layers into the meta-phosphor, and a build picks
> the bar.bb recipe, what will happen?

I believe what Brad is proposing actually fixes your issue.  The meta
layers essentially go away, and your examples become:

meta-phosphor/recipes-power/bar/bar.bb
meta-phosphor/recipes-x86-intel/bar/bar.bb

At which point you immediately would get a warning that you have
duplicated recipe names, and we as a community can figure out how best
to disambiguate the two recipes.

Re: moving meta-{openpower, x86, arm} content to meta-phosphor

[Ed Tanous]

On Thu, Aug 20, 2020 at 6:18 AM Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
>
> Fellow OpenBMCers
>
> Over time, I would like to do away with the processor arch layers e.g.
>
> meta-openpower, meta-arm, meta-x86.
>
> In hindsight meta-arm and meta-x86 might not have made much sense and
> should have been something like meta-x86-intel and meta-x86-amd perhaps
> - this is confirmed by the fact that there isn't any content in meta-
> x86.
>
> I propose the content simply go in meta-phosphor, and that we frame our
> thinking of meta-phosphor and OpenBMC as a project that supports any and
> all host processor architectures (as well as devices that aren't servers
> at all).  This results in several positive things:
>
> - Increased developer/maintainer awareness/cross pollination of other
> usage patterns (community building).
> - Differences are obvious, highlighting areas for improvement in the
> project.
> - Build time, cross arch incompatibilities become obvious (e.g. building
> images that support both Intel and AMD processors for example).
> - Improved time to understanding for newcomers - everything is one
> place.
> - Reduced (granted a small amount) layer configuration complexity for
> end users.

Sounds pretty reasonable.  I wonder if we should go even further, and
not even model the host processor differences in meta layers at all.
We would simply enable certain features/packages for certain machines.
That makes sure that we aren't turning on features for machines that
haven't been tested yet, but doesn't require a full move over if
someone wants to enable a new processor type on a system that didn't
already support it.  It also hopefully means that the "default" would
be to start with an existing application, and simply add support for a
given processor/architecture/feature rather than starting over from
scratch more of the time.
The other thing I would add is that it (hopefully) leads to more reuse
between configurations.  Things like "peci-pcie" might one day become
"Pcie-inventory", and simply support all processor inventory types
that it can.

>
> I'm not aware of any benefits to factoring things out into the different
> layers like we have today - if you are aware of something, please share.
>
> Getting more detailed, how would this look?  This series is an example:
>
> https://gerrit.openbmc-project.xyz/35759
>
> For projects that are truly host processor specific e.g. peci or occ
> support, we already have a recipes-x86 folder:

It should be noted, I wrote the above comment about peci-pcie before I
read this part.  Hah.

>
> https://github.com/openbmc/meta-phosphor/tree/master/recipes-x86
>
> I propose we allow the creation of additional folders using this
> convention e.g.
>
> - recipes-power
> - recipes-x86-amd (we might want to look at renaming recipes-x86 to
> recipes-x86-intel)

I no longer have a dog in this fight, but as a developer the above
sounds reasonable.

>
> Or even better IMO, we co-mingle these recipes as well based on the
> abstract function they provide for some of the same reasons I would like
> to move to a single layer - increased awareness of what your community
> peers are up to.
>

I should really learn to read the whole document before writing
comments :)  Sounds like we had similar thoughts.  +1