Managing openbmc with subtree

Managing openbmc with subtree

[Dave Cobbley]

Brad,
Trying to identify the appropriate work-flowfor using git subtree to 
maintain all the meta-data under openbmc.
I'm imagining something like:

17 subtrees
** indicates an actual folder, not subtree.
meta-openembedded/
meta-raspberrypi/
meta-virtualization/
poky/
**meta-openbmc-machines/
** meta-arm
    ├── common
    └── meta-qualcomm
**meta-evb
    ├── meta-evb-aspeed
    ├── meta-evb-nuvoton
    ├── meta-evb-raspberrypi
**meta-openpower
    ├── meta-ibm
    ├── meta-ingrasys
    ├── meta-inventec
    ├── meta-rackspace
**meta-x86
     ├── meta-intel
     ├── meta-mellanox
     ├── meta-portwell
     └── meta-quanta
**meta-openbmc-bsp/
     ├── meta-aspeed
     ├── meta-ibm
     ├── meta-nuvoton
meta-phosphor/


I began working on the workflows that a typical dev would do to check 
code in and have Jenkins automatically update the top level repo with 
the new additions, but run into troublewith a merge commit.

1. git clone openbmc-openbmc
2. #make changes
3. git add
4. git commit...
5. git subtree push --prefix=path/to/local/subtree 
ssh://openbmc.gerrit/openbmc/<subtree_name> HEAD:refs/for/master

At this point, the build CI would run it's normal operations and code 
review would take place.
Following a +2 and submit, Jenkins would:

1. cd openbmc-openbmc
2. git pull
3. git subtree pull --prefix=path/to/subtree --squash <remote> master 
#For all given subtrees

However at this point, I end up with two commits in the parent repo, one 
with the squashed changes, and a merge commit.
I don't know how to do this operation without ending up with a merge 
commit, do you have any advice here?

I could be totally overthinking the whole strategy, so any advice and 
direction is much appreciated!
Along those same lines, is subtree still the correct tool? I hear repo 
works for a lot of projects, Yocto uses 
https://wiki.yoctoproject.org/wiki/Combo-layer.

Thanks,
-Dave

Re: Managing openbmc with subtree

[Dave Cobbley]

The closest paradigm I can figure is this:

               +---------+                   | +-------------+
               |   Dev   |                   |               | Jenkins   |
               +---------+                   | +-------------+
+--------------------------------------------------------------------------------------------------------------+
    1. Modify subtree                        |
    2. git add/commit                        |
    3. git push gerrit HEAD:refs/for/master  |
                                             |   Once +2 is given
                                             |   4. git subtree pull 
--prefix=path/to/subtree <subtree> master
                                             |   5. git checkout <ref>
                                             |   6. git reset --soft ~1
                                             |   7. git commit <with 
message from original author>
                                             |   8. git add/commit
                                             |   9. git push
    10. git pull --rebase                    |
                                             |
                                             |
                                             |
                                             +

It seems wrong to do a reset --soft. Git must have a better way of doing 
this with subtrees.

Thanks,
-Dave


On 07/19/2018 01:08 PM, Dave Cobbley wrote:
> Brad,
> Trying to identify the appropriate work-flowfor using git subtree to 
> maintain all the meta-data under openbmc.
> I'm imagining something like:
>
> 17 subtrees
> ** indicates an actual folder, not subtree.
> meta-openembedded/
> meta-raspberrypi/
> meta-virtualization/
> poky/
> **meta-openbmc-machines/
> ** meta-arm
>    ├── common
>    └── meta-qualcomm
> **meta-evb
>    ├── meta-evb-aspeed
>    ├── meta-evb-nuvoton
>    ├── meta-evb-raspberrypi
> **meta-openpower
>    ├── meta-ibm
>    ├── meta-ingrasys
>    ├── meta-inventec
>    ├── meta-rackspace
> **meta-x86
>     ├── meta-intel
>     ├── meta-mellanox
>     ├── meta-portwell
>     └── meta-quanta
> **meta-openbmc-bsp/
>     ├── meta-aspeed
>     ├── meta-ibm
>     ├── meta-nuvoton
> meta-phosphor/
>
>
> I began working on the workflows that a typical dev would do to check 
> code in and have Jenkins automatically update the top level repo with 
> the new additions, but run into troublewith a merge commit.
>
> 1. git clone openbmc-openbmc
> 2. #make changes
> 3. git add
> 4. git commit...
> 5. git subtree push --prefix=path/to/local/subtree 
> ssh://openbmc.gerrit/openbmc/<subtree_name> HEAD:refs/for/master
>
> At this point, the build CI would run it's normal operations and code 
> review would take place.
> Following a +2 and submit, Jenkins would:
>
> 1. cd openbmc-openbmc
> 2. git pull
> 3. git subtree pull --prefix=path/to/subtree --squash <remote> master 
> #For all given subtrees
>
> However at this point, I end up with two commits in the parent repo, 
> one with the squashed changes, and a merge commit.
> I don't know how to do this operation without ending up with a merge 
> commit, do you have any advice here?
>
> I could be totally overthinking the whole strategy, so any advice and 
> direction is much appreciated!
> Along those same lines, is subtree still the correct tool? I hear repo 
> works for a lot of projects, Yocto uses 
> https://wiki.yoctoproject.org/wiki/Combo-layer.
>
> Thanks,
> -Dave

Re: Managing openbmc with subtree

[Brad Bishop]

> On Jul 19, 2018, at 4:08 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
> 
> Brad,
> Trying to identify the appropriate work-flowfor using git subtree to maintain all the meta-data under openbmc.
> I'm imagining something like:
> 
> 17 subtrees
> ** indicates an actual folder, not subtree.
> meta-openembedded/
> meta-raspberrypi/
> meta-virtualization/
> poky/
> **meta-openbmc-machines/
> ** meta-arm
>    ├── common
>    └── meta-qualcomm
> **meta-evb
>    ├── meta-evb-aspeed
>    ├── meta-evb-nuvoton
>    ├── meta-evb-raspberrypi
> **meta-openpower
>    ├── meta-ibm
>    ├── meta-ingrasys
>    ├── meta-inventec
>    ├── meta-rackspace
> **meta-x86
>     ├── meta-intel
>     ├── meta-mellanox
>     ├── meta-portwell
>     └── meta-quanta
> **meta-openbmc-bsp/
>     ├── meta-aspeed
>     ├── meta-ibm
>     ├── meta-nuvoton
> meta-phosphor/
> 

This is a great picture, thanks.  My longer-term vision is a bit more like this:

meta-openembedded/
meta-virtualization/
poky/
meta-arm/
meta-openpower/
meta-x86/
meta-phosphor/

Or even ditching Yocto and just using OE-Core/bitbake directly:

meta-openembedded/
meta-virtualization/
meta/
bitbake/
meta-arm/
meta-openpower/
meta-x86/
meta-phosphor/

I could understand a case for including the BSPs in the base distro:

meta-openembedded/
meta-virtualization/
meta/
bitbake/
meta-arm/
meta-openpower/
meta-x86/
meta-phosphor/
meta-aspeed/
meta-ibm/ -> for our SOCs only, not servers
meta-nuvoton/
meta-raspberrypi/

The main thing to notice with all of these is I’ve removed meta-foo from
meta-openpower, meta-x86 and meta-arm.  This lets people have arm, power
and x86 servers all in the same layer or repo if they want:

meta-xyzcorp/
   |— meta-arm-server/
   |- meta-x86-server/
   |- meta-power-server/

Actually I’ve removed them from the repository completely.  I think we
need some kind of reference platform(s) just like Yocto.  Otherwise we are
going to wind up with too many server-layers in the distro.

Another idea is to scrap meta-openpower, meta-arm and meta-x86
and put all that content in meta-phosphor:

meta-openembedded/
meta-virtualization/
meta/
bitbake/
meta-phosphor/

What do you think of all this?  There is a lot of change here, so some
staging would be required.  Obviously all ideas are debatable.

> 
> I began working on the workflows that a typical dev would do to check code in and have Jenkins automatically update the top level repo with the new additions, but run into troublewith a merge commit.
> 
> 1. git clone openbmc-openbmc
> 2. #make changes
> 3. git add
> 4. git commit...
> 5. git subtree push --prefix=path/to/local/subtree ssh://openbmc.gerrit/openbmc/<subtree_name> HEAD:refs/for/master

Looks good to me.  This should be pretty similar to the workflow
for OE-Core -> Poky right?

> 
> At this point, the build CI would run it's normal operations and code review would take place.
> Following a +2 and submit, Jenkins would:
> 
> 1. cd openbmc-openbmc
> 2. git pull
> 3. git subtree pull --prefix=path/to/subtree --squash <remote> master #For all given subtrees

I think I read somewhere that Poky has another verification and approval
step here, before merging OE-Core/bitbake patches into Poky.  Do we want
to be like Poky and require testing and/or maintainer approval here?

> 
> However at this point, I end up with two commits in the parent repo, one with the squashed changes, and a merge commit.
> I don't know how to do this operation without ending up with a merge commit, do you have any advice here?

To avoid the merge you can skip the subtree pull (which does a merge) and
instead generate a patch:

git cherry-pick —strategy=subtree -Xsubtree=path/to/subtree <squash-commit>

> 
> I could be totally overthinking the whole strategy,

I don’t think so.  Its par for the course I think for an OE based distribution.

> so any advice and direction is much appreciated!
> Along those same lines, is subtree still the correct tool?

I think so.  I had a look at Combo-layer awhile back…it seems to do the
same thing as git cherry-pick, that is, generate patches and apply them
to the parent repository.  It does layer some features on top of that
but I haven’t missed those yet.  They are similar enough I feel we could
switch back and forth if need be.  I’m guessing Combo-layer was written
before subtrees had the support they do today in Git.

> I hear repo works for a lot of projects, Yocto uses https://wiki.yoctoproject.org/wiki/Combo-layer.
> 
> Thanks,
> -Dave

Thanks for kicking off this conversation!

-brad

Re: Managing openbmc with subtree

[Brad Bishop]

> On Jul 20, 2018, at 1:43 PM, Brad Bishop <bradleyb@fuzziesquirrel.com> wrote:
>> On Jul 19, 2018, at 4:08 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>> 
>> However at this point, I end up with two commits in the parent repo, one with the squashed changes, and a merge commit.
>> I don't know how to do this operation without ending up with a merge commit, do you have any advice here?
> 
> To avoid the merge you can skip the subtree pull (which does a merge) and
> instead generate a patch:
> 
> git cherry-pick —strategy=subtree -Xsubtree=path/to/subtree <squash-commit>

Oops.  You’ll still have to do the pull and _then_ generate a patch and
discard the merge commit.  Have a look here:

https://github.com/openbmc/openbmc/blob/master/import-layers/HOWTO-update.md

Re: Managing openbmc with subtree

[Dave Cobbley]

I posted an initial draft of what permissions could look like in the top 
level meta-data.

https://gerrit.openbmc-project.xyz/#/c/openbmc/openbmc/+/11730/

This will require additional people receive +2 permissions, but the 
OWNERS plugin will ensure that a patch cannot get merged unless it has 
+2 from the appropriate people.

Thanks,
-Dave
>> On Jul 19, 2018, at 4:08 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>
>> Brad,
>> Trying to identify the appropriate work-flowfor using git subtree to maintain all the meta-data under openbmc.
>> I'm imagining something like:
>>
>> 17 subtrees
>> ** indicates an actual folder, not subtree.
>> meta-openembedded/
>> meta-raspberrypi/
>> meta-virtualization/
>> poky/
>> **meta-openbmc-machines/
>> ** meta-arm
>>     ├── common
>>     └── meta-qualcomm
>> **meta-evb
>>     ├── meta-evb-aspeed
>>     ├── meta-evb-nuvoton
>>     ├── meta-evb-raspberrypi
>> **meta-openpower
>>     ├── meta-ibm
>>     ├── meta-ingrasys
>>     ├── meta-inventec
>>     ├── meta-rackspace
>> **meta-x86
>>      ├── meta-intel
>>      ├── meta-mellanox
>>      ├── meta-portwell
>>      └── meta-quanta
>> **meta-openbmc-bsp/
>>      ├── meta-aspeed
>>      ├── meta-ibm
>>      ├── meta-nuvoton
>> meta-phosphor/
>>
> This is a great picture, thanks.  My longer-term vision is a bit more like this:
>
> meta-openembedded/
> meta-virtualization/
> poky/
> meta-arm/
> meta-openpower/
> meta-x86/
> meta-phosphor/
>
> Or even ditching Yocto and just using OE-Core/bitbake directly:
>
> meta-openembedded/
> meta-virtualization/
> meta/
> bitbake/
> meta-arm/
> meta-openpower/
> meta-x86/
> meta-phosphor/
>
> I could understand a case for including the BSPs in the base distro:
>
> meta-openembedded/
> meta-virtualization/
> meta/
> bitbake/
> meta-arm/
> meta-openpower/
> meta-x86/
> meta-phosphor/
> meta-aspeed/
> meta-ibm/ -> for our SOCs only, not servers
> meta-nuvoton/
> meta-raspberrypi/
>
> The main thing to notice with all of these is I’ve removed meta-foo from
> meta-openpower, meta-x86 and meta-arm.  This lets people have arm, power
> and x86 servers all in the same layer or repo if they want:
>
> meta-xyzcorp/
>     |— meta-arm-server/
>     |- meta-x86-server/
>     |- meta-power-server/
>
> Actually I’ve removed them from the repository completely.  I think we
> need some kind of reference platform(s) just like Yocto.  Otherwise we are
> going to wind up with too many server-layers in the distro.
>
> Another idea is to scrap meta-openpower, meta-arm and meta-x86
> and put all that content in meta-phosphor:
>
> meta-openembedded/
> meta-virtualization/
> meta/
> bitbake/
> meta-phosphor/
>
> What do you think of all this?  There is a lot of change here, so some
> staging would be required.  Obviously all ideas are debatable.
>
>> I began working on the workflows that a typical dev would do to check code in and have Jenkins automatically update the top level repo with the new additions, but run into troublewith a merge commit.
>>
>> 1. git clone openbmc-openbmc
>> 2. #make changes
>> 3. git add
>> 4. git commit...
>> 5. git subtree push --prefix=path/to/local/subtree ssh://openbmc.gerrit/openbmc/<subtree_name> HEAD:refs/for/master
> Looks good to me.  This should be pretty similar to the workflow
> for OE-Core -> Poky right?
>
>> At this point, the build CI would run it's normal operations and code review would take place.
>> Following a +2 and submit, Jenkins would:
>>
>> 1. cd openbmc-openbmc
>> 2. git pull
>> 3. git subtree pull --prefix=path/to/subtree --squash <remote> master #For all given subtrees
> I think I read somewhere that Poky has another verification and approval
> step here, before merging OE-Core/bitbake patches into Poky.  Do we want
> to be like Poky and require testing and/or maintainer approval here?
>
>> However at this point, I end up with two commits in the parent repo, one with the squashed changes, and a merge commit.
>> I don't know how to do this operation without ending up with a merge commit, do you have any advice here?
> To avoid the merge you can skip the subtree pull (which does a merge) and
> instead generate a patch:
>
> git cherry-pick —strategy=subtree -Xsubtree=path/to/subtree <squash-commit>
>
>> I could be totally overthinking the whole strategy,
> I don’t think so.  Its par for the course I think for an OE based distribution.
>
>> so any advice and direction is much appreciated!
>> Along those same lines, is subtree still the correct tool?
> I think so.  I had a look at Combo-layer awhile back…it seems to do the
> same thing as git cherry-pick, that is, generate patches and apply them
> to the parent repository.  It does layer some features on top of that
> but I haven’t missed those yet.  They are similar enough I feel we could
> switch back and forth if need be.  I’m guessing Combo-layer was written
> before subtrees had the support they do today in Git.
>
>> I hear repo works for a lot of projects, Yocto uses https://wiki.yoctoproject.org/wiki/Combo-layer.
>>
>> Thanks,
>> -Dave
> Thanks for kicking off this conversation!
>
> -brad

Re: Managing openbmc with subtree

[Ed Tanous]

+1

This looks like exactly what I'd like to see moving forward for the repo 
management as a whole.  individual meta layers can be broken out, and 
don't need to wait on merges to the top layer, which will likely take 
more time than simply applying it to a single layer or product.

Re: Managing openbmc with subtree

[Brad Bishop]

> On Aug 1, 2018, at 5:53 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
> 
> I posted an initial draft of what permissions could look like in the top level meta-data.
> 
> https://gerrit.openbmc-project.xyz/#/c/openbmc/openbmc/+/11730/

At first I was OK with this idea but instead of rushing to something
I’d like to slow down for a sec and be sure we don’t like the other
options.

My issue with 11730 is that we wind up with different workflows for
updating the different subtrees - for example updating meta-virtualization
is different than updating meta-phosphor.

If we go back to your original workflow where the developer pushes directly
to the subtree repository instead of openbmc/openbmc, recall we were stuck
on how to do it:

>>> However at this point, I end up with two commits in the parent repo, one with the squashed changes, and a merge commit.
>>> I don't know how to do this operation without ending up with a merge commit, do you have any advice here?
>> To avoid the merge you can skip the subtree pull (which does a merge) and
>> instead generate a patch:
>> 
>> git cherry-pick —strategy=subtree -Xsubtree=path/to/subtree <squash-commit>


Elaborating on my suggestion from before - here is a possible workflow.

Consider a repository with two remotes:

$ cat ~/.ssh/config
Host openbmc.gerrit
        Hostname gerrit.openbmc-project.xyz
        Port 29418
        User user
        IdentityFile ~/.ssh/my-gerrit-key

$ git remote -v
meta-phosphor-gerrit    ssh://openbmc.gerrit/openbmc/meta-phosphor (fetch)
meta-phosphor-gerrit    ssh://openbmc.gerrit/openbmc/meta-phosphor (push)
openbmc-gerrit  ssh://openbmc.gerrit/openbmc/openbmc (fetch)
openbmc-gerrit  ssh://openbmc.gerrit/openbmc/openbmc (push)

You can make and commit changes as usual.  When it comes time to push for
review:

$ git checkout -b phosphor-master meta-phosphor-gerrit/master
$ git cherry-pick —-strategy=subtree -Xsubtree=meta-phosphor <openbmc/openbmc-sha>
$ git push meta-phosphor-gerrit HEAD:refs/for/master

This gives us the same process for all subtrees, whether or not they are
openbmc hosted subtrees.  It also does not require people hacking on a
subtree to clone OpenBMC when it doesn’t make sense for them to do that,
such as someone just using the Aspeed BSP layer.

Granted it is a couple extra commands, but it seems like something that could
be scripted pretty easily.  Thoughts?

-brad

> 
> This will require additional people receive +2 permissions, but the OWNERS plugin will ensure that a patch cannot get merged unless it has +2 from the appropriate people.
> 
> Thanks,
> -Dave

Re: Managing openbmc with subtree

[Dave Cobbley]

I do agree that we can find a scriptable solution to this, however, I 
feel using the OWNERS plugin provides a much easier workflow for all 
developers, i.e. nothing changes from their perspective.

Additionally, we can still subtree on the back end after code is 
successfully merged into master.

Thanks,
-Dave

On 08/06/2018 11:47 AM, Brad Bishop wrote:
>> On Aug 1, 2018, at 5:53 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>
>> I posted an initial draft of what permissions could look like in the top level meta-data.
>>
>> https://gerrit.openbmc-project.xyz/#/c/openbmc/openbmc/+/11730/
> At first I was OK with this idea but instead of rushing to something
> I’d like to slow down for a sec and be sure we don’t like the other
> options.
>
> My issue with 11730 is that we wind up with different workflows for
> updating the different subtrees - for example updating meta-virtualization
> is different than updating meta-phosphor.
>
> If we go back to your original workflow where the developer pushes directly
> to the subtree repository instead of openbmc/openbmc, recall we were stuck
> on how to do it:
>
>>>> However at this point, I end up with two commits in the parent repo, one with the squashed changes, and a merge commit.
>>>> I don't know how to do this operation without ending up with a merge commit, do you have any advice here?
>>> To avoid the merge you can skip the subtree pull (which does a merge) and
>>> instead generate a patch:
>>>
>>> git cherry-pick —strategy=subtree -Xsubtree=path/to/subtree <squash-commit>
>
> Elaborating on my suggestion from before - here is a possible workflow.
>
> Consider a repository with two remotes:
>
> $ cat ~/.ssh/config
> Host openbmc.gerrit
>          Hostname gerrit.openbmc-project.xyz
>          Port 29418
>          User user
>          IdentityFile ~/.ssh/my-gerrit-key
>
> $ git remote -v
> meta-phosphor-gerrit    ssh://openbmc.gerrit/openbmc/meta-phosphor (fetch)
> meta-phosphor-gerrit    ssh://openbmc.gerrit/openbmc/meta-phosphor (push)
> openbmc-gerrit  ssh://openbmc.gerrit/openbmc/openbmc (fetch)
> openbmc-gerrit  ssh://openbmc.gerrit/openbmc/openbmc (push)
>
> You can make and commit changes as usual.  When it comes time to push for
> review:
>
> $ git checkout -b phosphor-master meta-phosphor-gerrit/master
> $ git cherry-pick —-strategy=subtree -Xsubtree=meta-phosphor <openbmc/openbmc-sha>
> $ git push meta-phosphor-gerrit HEAD:refs/for/master
>
> This gives us the same process for all subtrees, whether or not they are
> openbmc hosted subtrees.  It also does not require people hacking on a
> subtree to clone OpenBMC when it doesn’t make sense for them to do that,
> such as someone just using the Aspeed BSP layer.
>
> Granted it is a couple extra commands, but it seems like something that could
> be scripted pretty easily.  Thoughts?
>
> -brad
>
>> This will require additional people receive +2 permissions, but the OWNERS plugin will ensure that a patch cannot get merged unless it has +2 from the appropriate people.
>>
>> Thanks,
>> -Dave

Re: Managing openbmc with subtree

[Brad Bishop]

> On Aug 7, 2018, at 1:16 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
> 
> I do agree that we can find a scriptable solution to this, however, I feel using the OWNERS plugin provides a much easier workflow for all developers, i.e. nothing changes from their perspective.

Hrm, while I agree that the OWNERS plugin preserves the existing workflow
I’m not convinced that is the best path forward.

>> This gives us the same process for all subtrees, whether or not they are
>> openbmc hosted subtrees.  It also does not require people hacking on a
>> subtree to clone OpenBMC when it doesn’t make sense for them to do that,
>> such as someone just using the Aspeed BSP layer.


I still have these concerns.

Re: Managing openbmc with subtree

[Dave Cobbley]

>> On Aug 7, 2018, at 1:16 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>
>> I do agree that we can find a scriptable solution to this, however, I feel using the OWNERS plugin provides a much easier workflow for all developers, i.e. nothing changes from their perspective.
> Hrm, while I agree that the OWNERS plugin preserves the existing workflow
> I’m not convinced that is the best path forward.
I suppose it does only give us one of the two goals, where as moving 
straight to subtrees fulfills all of the goals.
>
>>> This gives us the same process for all subtrees, whether or not they are
>>> openbmc hosted subtrees.  It also does not require people hacking on a
>>> subtree to clone OpenBMC when it doesn’t make sense for them to do that,
>>> such as someone just using the Aspeed BSP layer.
>
> I still have these concerns.
I understand your concerns and agree - Here is what I would propose for 
the subtree folder structure:

Splat * represents a subtree

Yocto & more:
import-layers/
     *meta-openembedded/
     *meta-virtualization/
     *poky/
     *meta-security/

BSP:
meta-openbmc-bsp/
    *meta-ibm/
    *meta-aspeed/
    *meta-nuvoton/
    *meta-raspberrypi/

MACHINES:
meta-openbmc-machines/
    meta-arm/
        *meta-qualcomm/
    meta-openpower/
        * meta-ibm/
        * meta-ingrasys/
        * meta-inventec/
        * meta-rackspace/
    meta-x86/
        *meta-intel/
        *meta-mellanox/
        *meta-portwell/
        *meta-quanta/

REFERENCE:
meta-phosphor/


-Dave

Re: Managing openbmc with subtree

[Brad Bishop]

> On Aug 8, 2018, at 6:14 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
> 
> 
>>> On Aug 7, 2018, at 1:16 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>> 
>>> I do agree that we can find a scriptable solution to this, however, I feel using the OWNERS plugin provides a much easier workflow for all developers, i.e. nothing changes from their perspective.
>> Hrm, while I agree that the OWNERS plugin preserves the existing workflow
>> I’m not convinced that is the best path forward.
> I suppose it does only give us one of the two goals, where as moving straight to subtrees fulfills all of the goals.
>> 
>>>> This gives us the same process for all subtrees, whether or not they are
>>>> openbmc hosted subtrees.  It also does not require people hacking on a
>>>> subtree to clone OpenBMC when it doesn’t make sense for them to do that,
>>>> such as someone just using the Aspeed BSP layer.
>> 
>> I still have these concerns.
> I understand your concerns and agree

Is this an agreement to my proposed workflow?

For reference the proposal was distro hackers could have an openbmc/openbmc repo
with multiple remotes, one for the distro and one for each subtree and then:

>>>> $ git checkout -b phosphor-master meta-phosphor-gerrit/master
>>>> $ git cherry-pick —-strategy=subtree -Xsubtree=meta-phosphor <openbmc/openbmc-sha>
>>>> $ git push meta-phosphor-gerrit HEAD:refs/for/master
> 
> - Here is what I would propose for the subtree folder structure:
> 
> Splat * represents a subtree
> 
> Yocto & more:
> import-layers/
>     *meta-openembedded/
>     *meta-virtualization/
>     *poky/
>     *meta-security/

Would it make sense to put meta-* directly in the root, along with all our other
subtrees?  I don’t see the point in calling them out as upstream trees - I’d like
to see some or all of our project meta trees become hosted elsewhere someday,
much in the same way as meta-virt for example.

> 
> BSP:
> meta-openbmc-bsp/
>    *meta-ibm/
>    *meta-aspeed/
>    *meta-nuvoton/
>    *meta-raspberrypi/

lgtm

> 
> MACHINES:
> meta-openbmc-machines/
>    meta-arm/
>        *meta-qualcomm/
>    meta-openpower/
>        * meta-ibm/
>        * meta-ingrasys/
>        * meta-inventec/
>        * meta-rackspace/
>    meta-x86/
>        *meta-intel/
>        *meta-mellanox/
>        *meta-portwell/
>        *meta-quanta/

I’d like to get away from this and just put the company subtrees at the root.
That way meta-xyzcorp could support both an x86 and an arm server.

> 
> REFERENCE:
> meta-phosphor/

I’d suggest the arch metas here, at the root.  Honestly I’d like to get to a point
where all the arch layers are just merged into meta-phosphor and we can say
meta-phosphor supports x86, power, and arm servers (along with TORSes).  But we can
save that for another day.

> 
> 
> -Dave

So to net these all out:

*poky/

*meta-openembedded/
*meta-virtualization/
*meta-security/
*meta-arm/
*meta-openpower/
*meta-x86/
*meta-phosphor/

*meta-aspeed/
*meta-nuvoton/
*meta-raspberrypi/

*meta-intel/
*meta-mellanox/
*meta-portwell/
*meta-quanta/
*meta-ibm/
*meta-ingrasys/
*meta-inventec/
*meta-rackspace/
*meta-qualcomm/

Re: Managing openbmc with subtree

[Dave Cobbley]

On 08/09/2018 01:16 PM, Brad Bishop wrote:
>> On Aug 8, 2018, at 6:14 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>
>>
>>>> On Aug 7, 2018, at 1:16 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>>>
>>>> I do agree that we can find a scriptable solution to this, however, I feel using the OWNERS plugin provides a much easier workflow for all developers, i.e. nothing changes from their perspective.
>>> Hrm, while I agree that the OWNERS plugin preserves the existing workflow
>>> I’m not convinced that is the best path forward.
>> I suppose it does only give us one of the two goals, where as moving straight to subtrees fulfills all of the goals.
>>>>> This gives us the same process for all subtrees, whether or not they are
>>>>> openbmc hosted subtrees.  It also does not require people hacking on a
>>>>> subtree to clone OpenBMC when it doesn’t make sense for them to do that,
>>>>> such as someone just using the Aspeed BSP layer.
>>> I still have these concerns.
>> I understand your concerns and agree
> Is this an agreement to my proposed workflow?
I am running through the workflows locally to ensure the paradigm works 
as planned.
I have verified that the commit and subtree branch/push works - still 
working through how a dev would go work on something else, and then jump 
back into the feature to update the commit.
>
> For reference the proposal was distro hackers could have an openbmc/openbmc repo
> with multiple remotes, one for the distro and one for each subtree and then:
>
>>>>> $ git checkout -b phosphor-master meta-phosphor-gerrit/master
>>>>> $ git cherry-pick —-strategy=subtree -Xsubtree=meta-phosphor <openbmc/openbmc-sha>
>>>>> $ git push meta-phosphor-gerrit HEAD:refs/for/master
>> - Here is what I would propose for the subtree folder structure:
>>
>> Splat * represents a subtree
>>
>> Yocto & more:
>> import-layers/
>>      *meta-openembedded/
>>      *meta-virtualization/
>>      *poky/
>>      *meta-security/
> Would it make sense to put meta-* directly in the root, along with all our other
> subtrees?  I don’t see the point in calling them out as upstream trees - I’d like
> to see some or all of our project meta trees become hosted elsewhere someday,
> much in the same way as meta-virt for example.
>
Sure I believe that is fine. I like to put things in boxes so that habit 
is bleeding through :)
>> BSP:
>> meta-openbmc-bsp/
>>     *meta-ibm/
>>     *meta-aspeed/
>>     *meta-nuvoton/
>>     *meta-raspberrypi/
> lgtm
>
>> MACHINES:
>> meta-openbmc-machines/
>>     meta-arm/
>>         *meta-qualcomm/
>>     meta-openpower/
>>         * meta-ibm/
>>         * meta-ingrasys/
>>         * meta-inventec/
>>         * meta-rackspace/
>>     meta-x86/
>>         *meta-intel/
>>         *meta-mellanox/
>>         *meta-portwell/
>>         *meta-quanta/
> I’d like to get away from this and just put the company subtrees at the root.
> That way meta-xyzcorp could support both an x86 and an arm server.
Makes sense.
>
>> REFERENCE:
>> meta-phosphor/
> I’d suggest the arch metas here, at the root.  Honestly I’d like to get to a point
> where all the arch layers are just merged into meta-phosphor and we can say
> meta-phosphor supports x86, power, and arm servers (along with TORSes).  But we can
> save that for another day.
>
>>
>> -Dave
> So to net these all out:
>
> *poky/
>
> *meta-openembedded/
> *meta-virtualization/
> *meta-security/
> *meta-arm/
> *meta-openpower/
> *meta-x86/
> *meta-phosphor/
>
> *meta-aspeed/
> *meta-nuvoton/
> *meta-raspberrypi/
>
> *meta-intel/
> *meta-mellanox/
> *meta-portwell/
> *meta-quanta/
> *meta-ibm/
> *meta-ingrasys/
> *meta-inventec/
> *meta-rackspace/
> *meta-qualcomm/
Just to clarify - *meta-intel would contain our machine specific 
configurations for particular intel platforms, while *meta-x86 would 
contain image-like things that are generic to all x86 server platforms?
Not sure I understand why meta-x86 exists if 
meta-intel/mellanox/portwell/quanta are all on the top level. Meta-x86 
was simply a folder in my proposal.

-Dave

Re: Managing openbmc with subtree

[Dave Cobbley]

>> So to net these all out:
>>
>> *poky/
>>
>> *meta-openembedded/
>> *meta-virtualization/
>> *meta-security/
>> *meta-arm/
>> *meta-openpower/
>> *meta-x86/
>> *meta-phosphor/
>>
>> *meta-aspeed/
>> *meta-nuvoton/
>> *meta-raspberrypi/
>>
>> *meta-intel/
>> *meta-mellanox/
>> *meta-portwell/
>> *meta-quanta/
>> *meta-ibm/
>> *meta-ingrasys/
>> *meta-inventec/
>> *meta-rackspace/
>> *meta-qualcomm/
> Just to clarify - *meta-intel would contain our machine specific 
> configurations for particular intel platforms, while *meta-x86 would 
> contain image-like things that are generic to all x86 server platforms?
> Not sure I understand why meta-x86 exists if 
> meta-intel/mellanox/portwell/quanta are all on the top level. Meta-x86 
> was simply a folder in my proposal.
Looking at this further, I believe I understand the difference:
meta-intel is our actual baseboard products
meta-x86 is all things that relate specifically to baseboards that 
contain x86 processors.

Does that sound right?
> -Dave

Re: Managing openbmc with subtree

[Brad Bishop]

> On Aug 9, 2018, at 8:31 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
> 
>>> 
>>> So to net these all out:
>>> 
>>> *poky/
>>> 
>>> *meta-openembedded/
>>> *meta-virtualization/
>>> *meta-security/
>>> *meta-arm/
>>> *meta-openpower/
>>> *meta-x86/
>>> *meta-phosphor/
>>> 
>>> *meta-aspeed/
>>> *meta-nuvoton/
>>> *meta-raspberrypi/
>>> 
>>> *meta-intel/
>>> *meta-mellanox/
>>> *meta-portwell/
>>> *meta-quanta/
>>> *meta-ibm/
>>> *meta-ingrasys/
>>> *meta-inventec/
>>> *meta-rackspace/
>>> *meta-qualcomm/
>> Just to clarify - *meta-intel would contain our machine specific configurations for particular intel platforms, while *meta-x86 would contain image-like things that are generic to all x86 server platforms?
>> Not sure I understand why meta-x86 exists if meta-intel/mellanox/portwell/quanta are all on the top level. Meta-x86 was simply a folder in my proposal.
> Looking at this further, I believe I understand the difference:
> meta-intel is our actual baseboard products
> meta-x86 is all things that relate specifically to baseboards that contain x86 processors.
> 
> Does that sound right?

Yes, exactly.

>> -Dave

Re: Managing openbmc with subtree

[Dave Cobbley]

On 08/09/2018 08:17 PM, Brad Bishop wrote:
>> On Aug 9, 2018, at 8:31 PM, Dave Cobbley <david.j.cobbley@linux.intel.com> wrote:
>>
>>>> So to net these all out:
>>>>
>>>> *poky/
>>>>
>>>> *meta-openembedded/
>>>> *meta-virtualization/
>>>> *meta-security/
>>>> *meta-arm/
>>>> *meta-openpower/
>>>> *meta-x86/
>>>> *meta-phosphor/
>>>>
>>>> *meta-aspeed/
>>>> *meta-nuvoton/
>>>> *meta-raspberrypi/
>>>>
>>>> *meta-intel/
>>>> *meta-mellanox/
>>>> *meta-portwell/
>>>> *meta-quanta/
>>>> *meta-ibm/
>>>> *meta-ingrasys/
>>>> *meta-inventec/
>>>> *meta-rackspace/
>>>> *meta-qualcomm/
>>> Just to clarify - *meta-intel would contain our machine specific configurations for particular intel platforms, while *meta-x86 would contain image-like things that are generic to all x86 server platforms?
>>> Not sure I understand why meta-x86 exists if meta-intel/mellanox/portwell/quanta are all on the top level. Meta-x86 was simply a folder in my proposal.
>> Looking at this further, I believe I understand the difference:
>> meta-intel is our actual baseboard products
>> meta-x86 is all things that relate specifically to baseboards that contain x86 processors.
>>
>> Does that sound right?
> Yes, exactly.
>
>>> -Dave
Brad,
Can you make new github projects for the following? Just about 
everything else has a repo for subtrees.

meta-arm/
meta-x86/
meta-nuvoton/
meta-intel/
meta-mellanox/
meta-portwell/
meta-quanta/
meta-ibm/
meta-ingrasys/
meta-inventec/
meta-rackspace/

Subtress that already have a repo:
*poky/
*meta-openembedded/
*meta-security/
*meta-openpower/
*meta-phosphor/
*meta-aspeed/
*meta-raspberrypi/
*meta-qualcomm/ -- is this qualcomm-misc?

Should we need to rename qualcomm-mist to meta-qualcomm?

Additionally, looking at next steps for this:
- Modify the build script to run a full yocto build with each individual 
subtree review.
- Need to create a script which will merge code into the top level 
openbmc/openbmc
     Still trying to image what this script will look like, for example:
     $ git clone openbmc/openbmc
     $ git cherry-pick --strategy=subtree -Xsubtree=$SubtreePath $SubtreeSHA
     $ git push #does it need to push for review? I'm assuming at this 
point the build had already run on the subtree repo...
- How to handle existing reviews? Do we need a script to keep the 
subtrees up to date?

Are there any other steps I am missing?

Thanks,
-Dave