* Idea for bbappend / layer organization and naming convention @ 2017-08-14 19:36 Gunnar Andersson 2017-08-15 3:02 ` Bruce Ashfield 0 siblings, 1 reply; 6+ messages in thread From: Gunnar Andersson @ 2017-08-14 19:36 UTC (permalink / raw) To: yocto; +Cc: automotive-discussions@lists.linuxfoundation.org Yocto community, Triggered by the previous email request to yocto@ about best practices for layer organization I'm finally firing off this email for (hopefully) some feedback on an idea we first kicked around, discussed for rough consensus and then decided to implement in the Yocto based GDP project [1]. You can see it as a trial, anything can be changed, but so far so good... We adopted a somewhat novel (but actually not really unique, see inside) naming convention [2] for all modifications to components that belong to other layers. This convention shows what layers are being modified by the .bbappends by actually naming the layer it is (intending to**) modify. The initiative also aims to highlight and separate .bbappends (modifications) from uniquely added components in our project (new .bb files) in GDP. **Note that we are well aware this does NOT control bitbake behavior (as neither does the recipe-xxx/ directory convention), and is therefore just a guide. If the project is misconfigured, bitbake could do something different than the naming convention implies and it could even be misleading, but at least the *programmer intent* is shown clearly. I think it also makes .bbappends more explicitly visible and might trigger the idea: Do we really need this? Are we really modifying *that* adopted layer, and if so, why? To Russel who wrote the previous request for guidance - feel free to consider, but I won't recommend this directly since it's by no means an agreed best practice in all of the community. I'm rather asking for other experienced OE developers to consider and comment on the idea. For oldtimers it's probably easy to find this odd and just be negative against it since well, "standard practice", inertia, NIH and all that, but I'll stick my neck out anyway because it seems to solve some issues of organization and understanding at least for us (and I'm a sucker for some flamebait ;) If the intro is too formal and long, just look at the final examples. I think you'll get it. Feel free to ask. - Gunnar -- Gunnar Andersson <gandersson@genivi.org> Development Lead GENIVI Alliance [1] https://github.com/GENIVI/genivi-dev-platform [2] Naming strategy for bitbake extension layers: https://at.projects.genivi.org/wiki/x/w4Xk ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Idea for bbappend / layer organization and naming convention 2017-08-14 19:36 Idea for bbappend / layer organization and naming convention Gunnar Andersson @ 2017-08-15 3:02 ` Bruce Ashfield 2017-08-18 14:46 ` Gunnar Andersson 0 siblings, 1 reply; 6+ messages in thread From: Bruce Ashfield @ 2017-08-15 3:02 UTC (permalink / raw) To: Gunnar Andersson, yocto; +Cc: automotive-discussions@lists.linuxfoundation.org On 2017-08-14 3:36 PM, Gunnar Andersson wrote: > > Yocto community, > > Triggered by the previous email request to yocto@ about best practices for > layer organization I'm finally firing off this email for (hopefully) some > feedback on an idea we first kicked around, discussed for rough consensus > and then decided to implement in the Yocto based GDP project [1]. You can > see it as a trial, anything can be changed, but so far so good... > > We adopted a somewhat novel (but actually not really unique, see inside) > naming convention [2] for all modifications to components that belong to > other layers. > I've been using a similar naming/sorting mechanism in layers that I maintain for several years now. When you have a single layer that is carrying bbappends to many other layers, I find that it really helps keep everything separated and aids finding the original recipe. (that being said, recent work with layer index, etc, make it easier to locate the original recipe and any bbappends .. but that doesn't preclude keeping things organized in a layer). Cheers, Bruce > This convention shows what layers are being modified by the .bbappends by > actually naming the layer it is (intending to**) modify. The initiative > also aims to highlight and separate .bbappends (modifications) from uniquely > added components in our project (new .bb files) in GDP. > > **Note that we are well aware this does NOT control bitbake behavior (as > neither does the recipe-xxx/ directory convention), and is therefore just a > guide. If the project is misconfigured, bitbake could do something > different than the naming convention implies and it could even be > misleading, but at least the *programmer intent* is shown clearly. > > I think it also makes .bbappends more explicitly visible and might trigger > the idea: Do we really need this? Are we really modifying *that* adopted > layer, and if so, why? > > To Russel who wrote the previous request for guidance - feel free to > consider, but I won't recommend this directly since it's by no means an > agreed best practice in all of the community. I'm rather asking for other > experienced OE developers to consider and comment on the idea. > > For oldtimers it's probably easy to find this odd and just be negative > against it since well, "standard practice", inertia, NIH and all that, but > I'll stick my neck out anyway because it seems to solve some issues of > organization and understanding at least for us (and I'm a sucker for some > flamebait ;) > > If the intro is too formal and long, just look at the final examples. I > think you'll get it. Feel free to ask. > > > - Gunnar > > -- > Gunnar Andersson <gandersson@genivi.org> > Development Lead > GENIVI Alliance > > [1] https://github.com/GENIVI/genivi-dev-platform > [2] Naming strategy for bitbake extension layers: > https://at.projects.genivi.org/wiki/x/w4Xk > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Idea for bbappend / layer organization and naming convention 2017-08-15 3:02 ` Bruce Ashfield @ 2017-08-18 14:46 ` Gunnar Andersson 2017-08-18 18:47 ` Bruce Ashfield 0 siblings, 1 reply; 6+ messages in thread From: Gunnar Andersson @ 2017-08-18 14:46 UTC (permalink / raw) To: Bruce Ashfield, yocto; +Cc: automotive-discussions@lists.linuxfoundation.org On Mon, 2017-08-14 at 23:02 -0400, Bruce Ashfield wrote: > On 2017-08-14 3:36 PM, Gunnar Andersson wrote: > > > > Yocto community, > > > > Triggered by the previous email request to yocto@ about best practices > > for layer organization I'm finally firing off this email for (hopefully) > > some feedback on an idea we first kicked around, discussed for rough > > consensus and then decided to implement in the Yocto based GDP project > > [1]. You can see it as a trial, anything can be changed, but so far so > > good... > > > > We adopted a somewhat novel (but actually not really unique, see inside) > > naming convention [2] for all modifications to components that belong to > > other layers. > > > > I've been using a similar naming/sorting mechanism in layers that > I maintain for several years now. Great. Is it similar, or /exactly/ like this? I'm only asking because I think it would be useful to have some written down rules and if there are additional tweaks that will improve this proposal, I'd like to hear them. I suspected it might actually be somewhat of a common practice. > When you have a single layer that is carrying bbappends to many other > layers, I find that it really helps keep everything separated and aids > finding the original recipe. Yes, that is the idea. > > (that being said, recent work with layer index, etc, make it > easier to locate the original recipe and any bbappends .. but that > doesn't preclude keeping things organized in a layer). ... but I didn't get any more feedback than yours, so I'll just leave it where it is for now. Maybe this is not something the other Yocto community cares about, or similar practices are actually done in practice, but not written down. Or maybe everyone is OK with the state of mixing .bb and .bbappend files within the layers... So far I think the initial experience on our side is positive. It's something that needs to be looked after a bit (we have a few mistakes to fix). But since some directories will only have .bb files, and others only .bbappend files, it's therefore easy to script a warning system for anything that is out of place. Best Regards - Gunnar > Cheers, > > Bruce > [trimmed] > > -- > > Gunnar Andersson <gandersson@genivi.org> > > Development Lead > > GENIVI Alliance > > > > [1] https://github.com/GENIVI/genivi-dev-platform > > [2] Naming strategy for bitbake extension layers: > > https://at.projects.genivi.org/wiki/x/w4Xk ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Idea for bbappend / layer organization and naming convention 2017-08-18 14:46 ` Gunnar Andersson @ 2017-08-18 18:47 ` Bruce Ashfield 2017-08-18 19:35 ` Mark Hatle 0 siblings, 1 reply; 6+ messages in thread From: Bruce Ashfield @ 2017-08-18 18:47 UTC (permalink / raw) To: Gunnar Andersson, yocto; +Cc: automotive-discussions@lists.linuxfoundation.org On 08/18/2017 10:46 AM, Gunnar Andersson wrote: > > On Mon, 2017-08-14 at 23:02 -0400, Bruce Ashfield wrote: >> On 2017-08-14 3:36 PM, Gunnar Andersson wrote: >>> >>> Yocto community, >>> >>> Triggered by the previous email request to yocto@ about best practices >>> for layer organization I'm finally firing off this email for (hopefully) >>> some feedback on an idea we first kicked around, discussed for rough >>> consensus and then decided to implement in the Yocto based GDP project >>> [1]. You can see it as a trial, anything can be changed, but so far so >>> good... >>> >>> We adopted a somewhat novel (but actually not really unique, see inside) >>> naming convention [2] for all modifications to components that belong to >>> other layers. >>> >> >> I've been using a similar naming/sorting mechanism in layers that >> I maintain for several years now. > > Great. Is it similar, or /exactly/ like this? I'm only asking because I > think it would be useful to have some written down rules and if there are > additional tweaks that will improve this proposal, I'd like to hear them. > Exactly. The layer name and recipe-* structure is nested in a layer that is carrying bbappends. That way there's zero confusion about where the main recipe can be found. Bruce > I suspected it might actually be somewhat of a common practice. > >> When you have a single layer that is carrying bbappends to many other >> layers, I find that it really helps keep everything separated and aids >> finding the original recipe. > > Yes, that is the idea. > >> >> (that being said, recent work with layer index, etc, make it >> easier to locate the original recipe and any bbappends .. but that >> doesn't preclude keeping things organized in a layer). > > ... but I didn't get any more feedback than yours, so I'll just leave it > where it is for now. Maybe this is not something the other Yocto community > cares about, or similar practices are actually done in practice, but not > written down. Or maybe everyone is OK with the state of mixing .bb and > .bbappend files within the layers... > > So far I think the initial experience on our side is positive. It's > something that needs to be looked after a bit (we have a few mistakes to > fix). But since some directories will only have .bb files, and others only > .bbappend files, it's therefore easy to script a warning system for anything > that is out of place. > > Best Regards > - Gunnar > >> Cheers, >> >> Bruce >> > [trimmed] > >>> -- >>> Gunnar Andersson <gandersson@genivi.org> >>> Development Lead >>> GENIVI Alliance >>> >>> [1] https://github.com/GENIVI/genivi-dev-platform >>> [2] Naming strategy for bitbake extension layers: >>> https://at.projects.genivi.org/wiki/x/w4Xk > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Idea for bbappend / layer organization and naming convention 2017-08-18 18:47 ` Bruce Ashfield @ 2017-08-18 19:35 ` Mark Hatle 2017-08-20 19:02 ` Stephane Desneux 0 siblings, 1 reply; 6+ messages in thread From: Mark Hatle @ 2017-08-18 19:35 UTC (permalink / raw) To: Bruce Ashfield, Gunnar Andersson, yocto Cc: automotive-discussions@lists.linuxfoundation.org On 8/18/17 1:47 PM, Bruce Ashfield wrote: > On 08/18/2017 10:46 AM, Gunnar Andersson wrote: >> >> On Mon, 2017-08-14 at 23:02 -0400, Bruce Ashfield wrote: >>> On 2017-08-14 3:36 PM, Gunnar Andersson wrote: >>>> >>>> Yocto community, >>>> >>>> Triggered by the previous email request to yocto@ about best practices >>>> for layer organization I'm finally firing off this email for (hopefully) >>>> some feedback on an idea we first kicked around, discussed for rough >>>> consensus and then decided to implement in the Yocto based GDP project >>>> [1]. You can see it as a trial, anything can be changed, but so far so >>>> good... >>>> >>>> We adopted a somewhat novel (but actually not really unique, see inside) >>>> naming convention [2] for all modifications to components that belong to >>>> other layers. >>>> >>> >>> I've been using a similar naming/sorting mechanism in layers that >>> I maintain for several years now. >> >> Great. Is it similar, or /exactly/ like this? I'm only asking because I >> think it would be useful to have some written down rules and if there are >> additional tweaks that will improve this proposal, I'd like to hear them. >> I suspect suggestions for best practices along with examples are what people would like to see. This may be reasonable to do with both the Yocto Project and OpenEmbedded communities. Obviously there are currently a large number of existing layers. So showing how this works, and examples of how it makes it easier to maintain and manage the work over the longer term is a really good idea. (Keep in mind, many of the current layers were originally designed/updated before there was a large ecosystem of layers. So a lot of poor examples, of this kind of maintenance, may exist. It is a good idea to steer people away from using these as examples, and even a way to potentially help the maintainers of those layers evolve into a better and more efficient format.) --Mark > Exactly. The layer name and recipe-* structure is nested in a layer > that is carrying bbappends. That way there's zero confusion about where > the main recipe can be found. > > Bruce > >> I suspected it might actually be somewhat of a common practice. >> >>> When you have a single layer that is carrying bbappends to many other >>> layers, I find that it really helps keep everything separated and aids >>> finding the original recipe. >> >> Yes, that is the idea. >> >>> >>> (that being said, recent work with layer index, etc, make it >>> easier to locate the original recipe and any bbappends .. but that >>> doesn't preclude keeping things organized in a layer). >> >> ... but I didn't get any more feedback than yours, so I'll just leave it >> where it is for now. Maybe this is not something the other Yocto community >> cares about, or similar practices are actually done in practice, but not >> written down. Or maybe everyone is OK with the state of mixing .bb and >> .bbappend files within the layers... >> >> So far I think the initial experience on our side is positive. It's >> something that needs to be looked after a bit (we have a few mistakes to >> fix). But since some directories will only have .bb files, and others only >> .bbappend files, it's therefore easy to script a warning system for anything >> that is out of place. >> >> Best Regards >> - Gunnar >> >>> Cheers, >>> >>> Bruce >>> >> [trimmed] >> >>>> -- >>>> Gunnar Andersson <gandersson@genivi.org> >>>> Development Lead >>>> GENIVI Alliance >>>> >>>> [1] https://github.com/GENIVI/genivi-dev-platform >>>> [2] Naming strategy for bitbake extension layers: >>>> https://at.projects.genivi.org/wiki/x/w4Xk >> >> > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Idea for bbappend / layer organization and naming convention 2017-08-18 19:35 ` Mark Hatle @ 2017-08-20 19:02 ` Stephane Desneux 0 siblings, 0 replies; 6+ messages in thread From: Stephane Desneux @ 2017-08-20 19:02 UTC (permalink / raw) To: Mark Hatle, Bruce Ashfield, Gunnar Andersson, yocto Cc: automotive-discussions@lists.linuxfoundation.org Hi all, I've followed this thread with interest. I'll try to bring here some ideas we had in parallel on Automotive Grade Linux (AGL) side. What we plan to do with backports on AGL is still WIP (see https://jira.automotivelinux.org/browse/SPEC-802). It's still cooking, but we're going to adopt a solution close to the one proposed by Gunnar in GENIVI: I mean: we agree on the same goal, but what we end up with differs slightly. The scheme we're thinking about would be typically something like this: meta-agl/meta-agl-backports/<meta-origin>/<recipes-origin>/<recipe>/<feature|reason>/<recipe>.bbappend Where: * the first meta-agl is our main git repo (not a meta in fact): the real meta is meta-agl/meta-agl :) * meta-agl/meta-agl-backports is a layer * <meta-origin> is the name of the layer on which the backport is applied * <recipes-origin> is the origin folder name for recipes in the origin layer * <recipe> is the name of the recipe which is backported/amended * <feature|reason> is something that defines why we have a backport here (bug id, quick desc...) And finally, in the folder where we put the bbappend, we want to add a README.backport in YAML, similar to this one: — Origin: <name> Origin-Url: <source repo url> Origin-Ref: <rev/tag> Destination: <AGL version name: dab, eel,...> Expiration: <when> (if known) Bug-AGL: <reference to bug ID> Upstream-Status: <the same as OE's Upstream-Status tag> Reason: <comment on multiple lines> — With that model, we expect to smooth the workflow and make the alignment on Yocto/OE/Poky versions easier. HTH Best regards, --- Stephane Desneux - CTO - IoT.bzh stephane.desneux@iot.bzh - www.iot.bzh On 18/08/17 21:35, Mark Hatle wrote: > On 8/18/17 1:47 PM, Bruce Ashfield wrote: >> On 08/18/2017 10:46 AM, Gunnar Andersson wrote: >>> >>> On Mon, 2017-08-14 at 23:02 -0400, Bruce Ashfield wrote: >>>> On 2017-08-14 3:36 PM, Gunnar Andersson wrote: >>>>> >>>>> Yocto community, >>>>> >>>>> Triggered by the previous email request to yocto@ about best practices >>>>> for layer organization I'm finally firing off this email for (hopefully) >>>>> some feedback on an idea we first kicked around, discussed for rough >>>>> consensus and then decided to implement in the Yocto based GDP project >>>>> [1]. You can see it as a trial, anything can be changed, but so far so >>>>> good... >>>>> >>>>> We adopted a somewhat novel (but actually not really unique, see inside) >>>>> naming convention [2] for all modifications to components that belong to >>>>> other layers. >>>>> >>>> >>>> I've been using a similar naming/sorting mechanism in layers that >>>> I maintain for several years now. >>> >>> Great. Is it similar, or /exactly/ like this? I'm only asking because I >>> think it would be useful to have some written down rules and if there are >>> additional tweaks that will improve this proposal, I'd like to hear them. >>> > > I suspect suggestions for best practices along with examples are what people > would like to see. This may be reasonable to do with both the Yocto Project and > OpenEmbedded communities. > > Obviously there are currently a large number of existing layers. So showing how > this works, and examples of how it makes it easier to maintain and manage the > work over the longer term is a really good idea. > > (Keep in mind, many of the current layers were originally designed/updated > before there was a large ecosystem of layers. So a lot of poor examples, of > this kind of maintenance, may exist. It is a good idea to steer people away > from using these as examples, and even a way to potentially help the maintainers > of those layers evolve into a better and more efficient format.) > > --Mark > >> Exactly. The layer name and recipe-* structure is nested in a layer >> that is carrying bbappends. That way there's zero confusion about where >> the main recipe can be found. >> >> Bruce >> >>> I suspected it might actually be somewhat of a common practice. >>> >>>> When you have a single layer that is carrying bbappends to many other >>>> layers, I find that it really helps keep everything separated and aids >>>> finding the original recipe. >>> >>> Yes, that is the idea. >>> >>>> >>>> (that being said, recent work with layer index, etc, make it >>>> easier to locate the original recipe and any bbappends .. but that >>>> doesn't preclude keeping things organized in a layer). >>> >>> ... but I didn't get any more feedback than yours, so I'll just leave it >>> where it is for now. Maybe this is not something the other Yocto community >>> cares about, or similar practices are actually done in practice, but not >>> written down. Or maybe everyone is OK with the state of mixing .bb and >>> .bbappend files within the layers... >>> >>> So far I think the initial experience on our side is positive. It's >>> something that needs to be looked after a bit (we have a few mistakes to >>> fix). But since some directories will only have .bb files, and others only >>> .bbappend files, it's therefore easy to script a warning system for anything >>> that is out of place. >>> >>> Best Regards >>> - Gunnar >>> >>>> Cheers, >>>> >>>> Bruce >>>> >>> [trimmed] >>> >>>>> -- >>>>> Gunnar Andersson <gandersson@genivi.org> >>>>> Development Lead >>>>> GENIVI Alliance >>>>> >>>>> [1] https://github.com/GENIVI/genivi-dev-platform >>>>> [2] Naming strategy for bitbake extension layers: >>>>> https://at.projects.genivi.org/wiki/x/w4Xk >>> >>> >> > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-08-20 19:02 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-08-14 19:36 Idea for bbappend / layer organization and naming convention Gunnar Andersson 2017-08-15 3:02 ` Bruce Ashfield 2017-08-18 14:46 ` Gunnar Andersson 2017-08-18 18:47 ` Bruce Ashfield 2017-08-18 19:35 ` Mark Hatle 2017-08-20 19:02 ` Stephane Desneux
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.