* Transitioning to Yocto, some basic questions about workflow
@ 2015-11-10 22:27 Michael Habibi
2015-11-11 0:06 ` Randy Witt
2015-11-11 14:00 ` Burton, Ross
0 siblings, 2 replies; 7+ messages in thread
From: Michael Habibi @ 2015-11-10 22:27 UTC (permalink / raw)
To: yocto
[-- Attachment #1: Type: text/plain, Size: 3522 bytes --]
All,
First I appreciate taking the time to read through this email. I have spent
the last couple weeks reading through Yocto/bitbake documentation, and have
run a couple of the reference builds, e.g. core-image-minimal. I have a
better-than-vague sense now of how it’s parsing confs/recipes/classes and
putting that together.
My understanding is that Yocto is nothing more than a reference build that
combines OE and BitBake to give us a good starting point for a custom
distribution. Hopefully that assumption is correct.
Our current build distribution at work is a monstrosity. We use a
proprietary distribution that combines their utilities that they provide,
along with various other open source utilities and the linux kernel.
Overall our source tree is on the order of several gig in size, with a
couple hundred build components. The build framework is close to buildroot
in that we just iterate through directories, build the applications (in
some cases it's source, in other cases it's a patched tarball), and build
up a root filesystem image that we install on the embedded target. We have
many challenges with this, namely with dependencies and build efficiencies
- we can't run parallel builds because the way it iterates through the
makefiles is not dependency-based, but directory-based.
This is fundamentally why I have been looking into Yocto/BB/OE as a
potential replacement distribution. However, I have a couple questions
stemming from my research. We can leverage the existing Yocto build for
various open source utilities where it pulls from the web, patches, builds,
and installs into the deployable image. We would probably want to colocate
the tarballs locally, because we would like to prevent people from having
to fetch from the web during build cycles.
I see that Yocto thinks of the deployable image and applications as
separate entities: first you'd build the distribution, then you would use
ADT/cross-toolchain/etc to build the applications and install them
separately. However, in our environment, we would need all of our custom IP
applications to be built as part of the deployable image, and not as a
separate procdure. For example, if I were to call "bitbake our-image", I
would like the deployable image to contain 1) the kernel, 2) various core
utilities and libraries for booting, and 3) our custom applications for our
device.
This means that somewhere in the yocto framework, we'd ideally have some
source code somewhere that would also be compiled via recipes/classes that
we'd have to custom write. Is there a best practice for this kind of
workflow? I don't mind not having source checked in to our VCS for things
like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
server), but we likely wouldn't want to host tarballs for the applications
we are writing and modifying day-to-day by dozens of engineers. Is there a
place where this source would best fit? Would it be under build/tmp/etc, or
perhaps we can locate the source under a layer directory, like
meta/source/our-ip-applications?
Or perhaps this workflow is just not recommended or supported by the Yocto
Project? If that's the case, does Yocto only recommend building the
distribution then building the applications as completely separate
workflows? Or is there another workflow that I haven't stumbled across yet?
Thanks again for all your help, and let me know if I can help clarify
anything,
Michael
[-- Attachment #2: Type: text/html, Size: 4478 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-10 22:27 Transitioning to Yocto, some basic questions about workflow Michael Habibi
@ 2015-11-11 0:06 ` Randy Witt
2015-11-11 1:10 ` Michael Habibi
` (2 more replies)
2015-11-11 14:00 ` Burton, Ross
1 sibling, 3 replies; 7+ messages in thread
From: Randy Witt @ 2015-11-11 0:06 UTC (permalink / raw)
To: Michael Habibi, yocto
Hi Michael,
See my replies below.
On 11/10/2015 02:27 PM, Michael Habibi wrote:
>
>
> This is fundamentally why I have been looking into Yocto/BB/OE as a
> potential replacement distribution. However, I have a couple questions
> stemming from my research. We can leverage the existing Yocto build for
> various open source utilities where it pulls from the web, patches, builds,
> and installs into the deployable image. We would probably want to colocate
> the tarballs locally, because we would like to prevent people from having
> to fetch from the web during build cycles.
>
This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local
mirrors to try before using the upstream url in the recipe. So each person
building would at least have to fetch from the local mirror for a build, but
subsequent builds wouldn't require fetching again as long as the DL_DIR was
preserved.
https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>
> I see that Yocto thinks of the deployable image and applications as
> separate entities: first you'd build the distribution, then you would use
> ADT/cross-toolchain/etc to build the applications and install them
> separately. However, in our environment, we would need all of our custom IP
> applications to be built as part of the deployable image, and not as a
> separate procdure. For example, if I were to call "bitbake our-image", I
> would like the deployable image to contain 1) the kernel, 2) various core
> utilities and libraries for booting, and 3) our custom applications for our
> device.
>
>
> This means that somewhere in the yocto framework, we'd ideally have some
> source code somewhere that would also be compiled via recipes/classes that
> we'd have to custom write. Is there a best practice for this kind of
> workflow? I don't mind not having source checked in to our VCS for things
> like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
> server), but we likely wouldn't want to host tarballs for the applications
> we are writing and modifying day-to-day by dozens of engineers. Is there a
> place where this source would best fit? Would it be under build/tmp/etc, or
> perhaps we can locate the source under a layer directory, like
> meta/source/our-ip-applications?
>
For all the internal applications at my previous employer we had a layer for all
internal items, it's quite common.
And if you're not aware, source code is not restricted to tarballs. You can use
git repos, svn and any mechanism the fetcher supports.(even local directories)
The following URL lists the types the fetcher supports.
https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
The workflow now encouraged, is to use recipetool to assist in creating recipes.
And then once the recipes are created, you can use devtool to actually do the
iterative development on the application represented by the recipe.
https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>
> Or perhaps this workflow is just not recommended or supported by the Yocto
> Project? If that's the case, does Yocto only recommend building the
> distribution then building the applications as completely separate
> workflows? Or is there another workflow that I haven't stumbled across yet?
>
There is nothing "wrong" with building the image each time as a developer.
However, it is time consuming to construct the full image and deploy it.
A much more appealing mechanism would be to use devtool to build the recipe you
are working on, and then use "devtool deploy-target" to then deploy the output
to a live machine. This would require a writeable filesystem on the target
device, but if that is available, the workflow and turnaround time is much faster.
>
> Thanks again for all your help, and let me know if I can help clarify
> anything,
>
>
> Michael
>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-11 0:06 ` Randy Witt
@ 2015-11-11 1:10 ` Michael Habibi
2015-11-11 2:33 ` Bob Cochran
2015-11-11 15:36 ` Michael Habibi
2 siblings, 0 replies; 7+ messages in thread
From: Michael Habibi @ 2015-11-11 1:10 UTC (permalink / raw)
To: Randy Witt; +Cc: yocto
[-- Attachment #1: Type: text/plain, Size: 4301 bytes --]
Thanks for the great info Randy.
On Tue, Nov 10, 2015 at 6:06 PM, Randy Witt <randy.e.witt@linux.intel.com>
wrote:
> Hi Michael,
> See my replies below.
> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>>
>>
>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>> potential replacement distribution. However, I have a couple questions
>> stemming from my research. We can leverage the existing Yocto build for
>> various open source utilities where it pulls from the web, patches, builds,
>> and installs into the deployable image. We would probably want to colocate
>> the tarballs locally, because we would like to prevent people from having
>> to fetch from the web during build cycles.
>>
> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local
> mirrors to try before using the upstream url in the recipe. So each person
> building would at least have to fetch from the local mirror for a build, but
> subsequent builds wouldn't require fetching again as long as the DL_DIR was
> preserved.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>>
>> I see that Yocto thinks of the deployable image and applications as
>> separate entities: first you'd build the distribution, then you would use
>> ADT/cross-toolchain/etc to build the applications and install them
>> separately. However, in our environment, we would need all of our custom IP
>> applications to be built as part of the deployable image, and not as a
>> separate procdure. For example, if I were to call "bitbake our-image", I
>> would like the deployable image to contain 1) the kernel, 2) various core
>> utilities and libraries for booting, and 3) our custom applications for our
>> device.
>>
>>
>> This means that somewhere in the yocto framework, we'd ideally have some
>> source code somewhere that would also be compiled via recipes/classes that
>> we'd have to custom write. Is there a best practice for this kind of
>> workflow? I don't mind not having source checked in to our VCS for things
>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
>> server), but we likely wouldn't want to host tarballs for the applications
>> we are writing and modifying day-to-day by dozens of engineers. Is there a
>> place where this source would best fit? Would it be under build/tmp/etc, or
>> perhaps we can locate the source under a layer directory, like
>> meta/source/our-ip-applications?
>>
> For all the internal applications at my previous employer we had a layer for all
> internal items, it's quite common.
> And if you're not aware, source code is not restricted to tarballs. You can use
> git repos, svn and any mechanism the fetcher supports.(even local directories)
> The following URL lists the types the fetcher supports.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
> The workflow now encouraged, is to use recipetool to assist in creating recipes.
> And then once the recipes are created, you can use devtool to actually do the
> iterative development on the application represented by the recipe.
> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>>
>> Or perhaps this workflow is just not recommended or supported by the Yocto
>> Project? If that's the case, does Yocto only recommend building the
>> distribution then building the applications as completely separate
>> workflows? Or is there another workflow that I haven't stumbled across yet?
>>
> There is nothing "wrong" with building the image each time as a developer.
> However, it is time consuming to construct the full image and deploy it.
> A much more appealing mechanism would be to use devtool to build the recipe you
> are working on, and then use "devtool deploy-target" to then deploy the output
> to a live machine. This would require a writeable filesystem on the target
> device, but if that is available, the workflow and turnaround time is much faster.
>>
>> Thanks again for all your help, and let me know if I can help clarify
>> anything,
>>
>>
>> Michael
>>
>>
>>
[-- Attachment #2: Type: text/html, Size: 5024 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-11 0:06 ` Randy Witt
2015-11-11 1:10 ` Michael Habibi
@ 2015-11-11 2:33 ` Bob Cochran
2015-11-11 15:36 ` Michael Habibi
2 siblings, 0 replies; 7+ messages in thread
From: Bob Cochran @ 2015-11-11 2:33 UTC (permalink / raw)
To: Randy Witt, Michael Habibi, yocto
On 11/10/2015 07:06 PM, Randy Witt wrote:
> Hi Michael,
>
> See my replies below.
>
> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>
>>
>>
>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>> potential replacement distribution. However, I have a couple questions
>> stemming from my research. We can leverage the existing Yocto build for
>> various open source utilities where it pulls from the web, patches,
>> builds,
>> and installs into the deployable image. We would probably want to
>> colocate
>> the tarballs locally, because we would like to prevent people from having
>> to fetch from the web during build cycles.
>>
>
> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify
> local mirrors to try before using the upstream url in the recipe. So
> each person building would at least have to fetch from the local mirror
> for a build, but subsequent builds wouldn't require fetching again as
> long as the DL_DIR was preserved.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>
>
>>
>> I see that Yocto thinks of the deployable image and applications as
>> separate entities: first you'd build the distribution, then you would use
>> ADT/cross-toolchain/etc to build the applications and install them
>> separately. However, in our environment, we would need all of our
>> custom IP
>> applications to be built as part of the deployable image, and not as a
>> separate procdure. For example, if I were to call "bitbake our-image", I
>> would like the deployable image to contain 1) the kernel, 2) various core
>> utilities and libraries for booting, and 3) our custom applications
>> for our
>> device.
>>
>>
>> This means that somewhere in the yocto framework, we'd ideally have some
>> source code somewhere that would also be compiled via recipes/classes
>> that
>> we'd have to custom write. Is there a best practice for this kind of
>> workflow? I don't mind not having source checked in to our VCS for things
>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a
>> local
>> server), but we likely wouldn't want to host tarballs for the
>> applications
>> we are writing and modifying day-to-day by dozens of engineers. Is
>> there a
>> place where this source would best fit? Would it be under
>> build/tmp/etc, or
>> perhaps we can locate the source under a layer directory, like
>> meta/source/our-ip-applications?
>>
>
> For all the internal applications at my previous employer we had a layer
> for all internal items, it's quite common.
>
> And if you're not aware, source code is not restricted to tarballs. You
> can use git repos, svn and any mechanism the fetcher supports.(even
> local directories) The following URL lists the types the fetcher supports.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
I just wanted to add that it's also useful during prototyping to create
and install a toolchain built by poky onto the target's disk ( for us,
we'll have a small SSD attached to the target when doing this). This
can be great for quickly building up applications / daemons ( using C,
python, etc) that don't yet have recipes.
The tools are in the rootfs and the applications that we're prototyping
are on a separate partition. Even though it's prototype code, we run
git on the target and keep the code under SCM from the beginning.
Ultimately, our recipe will pull from the same git repo once the code
has matured and a recipe exists.
We add packages like "make binutils gcc gdb git" to our IMAGE_INSTALL
when building an image for our development disk / rootfs.
>
>
> The workflow now encouraged, is to use recipetool to assist in creating
> recipes. And then once the recipes are created, you can use devtool to
> actually do the iterative development on the application represented by
> the recipe.
> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>
>
>>
>> Or perhaps this workflow is just not recommended or supported by the
>> Yocto
>> Project? If that's the case, does Yocto only recommend building the
>> distribution then building the applications as completely separate
>> workflows? Or is there another workflow that I haven't stumbled across
>> yet?
>>
>
> There is nothing "wrong" with building the image each time as a
> developer. However, it is time consuming to construct the full image and
> deploy it.
>
> A much more appealing mechanism would be to use devtool to build the
> recipe you are working on, and then use "devtool deploy-target" to then
> deploy the output to a live machine. This would require a writeable
> filesystem on the target device, but if that is available, the workflow
> and turnaround time is much faster.
>
>>
>> Thanks again for all your help, and let me know if I can help clarify
>> anything,
>>
>>
>> Michael
>>
>>
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-10 22:27 Transitioning to Yocto, some basic questions about workflow Michael Habibi
2015-11-11 0:06 ` Randy Witt
@ 2015-11-11 14:00 ` Burton, Ross
1 sibling, 0 replies; 7+ messages in thread
From: Burton, Ross @ 2015-11-11 14:00 UTC (permalink / raw)
To: Michael Habibi; +Cc: yocto@yoctoproject.org
[-- Attachment #1: Type: text/plain, Size: 960 bytes --]
On 10 November 2015 at 22:27, Michael Habibi <mikehabibi@gmail.com> wrote:
> I see that Yocto thinks of the deployable image and applications as
> separate entities: first you'd build the distribution, then you would use
> ADT/cross-toolchain/etc to build the applications and install them
> separately. However, in our environment, we would need all of our custom IP
> applications to be built as part of the deployable image, and not as a
> separate procdure. For example, if I were to call "bitbake our-image", I
> would like the deployable image to contain 1) the kernel, 2) various core
> utilities and libraries for booting, and 3) our custom applications for our
> device.
Your observation here is incorrect: the ADT/etc are for developers to use
when developing the applications. If you're doing integration (i.e.
building images) then your applications are just more recipes to build
packages from that go into the image.
Ross
[-- Attachment #2: Type: text/html, Size: 1295 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-11 0:06 ` Randy Witt
2015-11-11 1:10 ` Michael Habibi
2015-11-11 2:33 ` Bob Cochran
@ 2015-11-11 15:36 ` Michael Habibi
2015-11-11 18:04 ` Khem Raj
2 siblings, 1 reply; 7+ messages in thread
From: Michael Habibi @ 2015-11-11 15:36 UTC (permalink / raw)
To: Randy Witt; +Cc: yocto
[-- Attachment #1: Type: text/plain, Size: 5018 bytes --]
Randy, I've had some time to think about your post and I had a follow up
question. Let's say we had internal applications, you suggest that the
fetcher can be used to pull down the git/svn repos that contains the
internal application code. In this environment, where is the Yocto
framework hosted? Is it also on a repo? Would then we have a Yocto repo for
image building, and an application repo for application development? Is
there a clean way of combining both Yocto and internal applications into a
single repo, so our engineers simply have to do a 'git clone' and they'll
have what they need to get started?
Thanks, still trying to wrap my head around this a bit.
On Tue, Nov 10, 2015 at 6:06 PM, Randy Witt <randy.e.witt@linux.intel.com>
wrote:
> Hi Michael,
>
> See my replies below.
>
> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>
>
>>
>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>> potential replacement distribution. However, I have a couple questions
>> stemming from my research. We can leverage the existing Yocto build for
>> various open source utilities where it pulls from the web, patches,
>> builds,
>> and installs into the deployable image. We would probably want to colocate
>> the tarballs locally, because we would like to prevent people from having
>> to fetch from the web during build cycles.
>>
>>
> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local
> mirrors to try before using the upstream url in the recipe. So each person
> building would at least have to fetch from the local mirror for a build,
> but subsequent builds wouldn't require fetching again as long as the DL_DIR
> was preserved.
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>
>
>> I see that Yocto thinks of the deployable image and applications as
>> separate entities: first you'd build the distribution, then you would use
>> ADT/cross-toolchain/etc to build the applications and install them
>> separately. However, in our environment, we would need all of our custom
>> IP
>> applications to be built as part of the deployable image, and not as a
>> separate procdure. For example, if I were to call "bitbake our-image", I
>> would like the deployable image to contain 1) the kernel, 2) various core
>> utilities and libraries for booting, and 3) our custom applications for
>> our
>> device.
>>
>>
>> This means that somewhere in the yocto framework, we'd ideally have some
>> source code somewhere that would also be compiled via recipes/classes that
>> we'd have to custom write. Is there a best practice for this kind of
>> workflow? I don't mind not having source checked in to our VCS for things
>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
>> server), but we likely wouldn't want to host tarballs for the applications
>> we are writing and modifying day-to-day by dozens of engineers. Is there a
>> place where this source would best fit? Would it be under build/tmp/etc,
>> or
>> perhaps we can locate the source under a layer directory, like
>> meta/source/our-ip-applications?
>>
>>
> For all the internal applications at my previous employer we had a layer
> for all internal items, it's quite common.
>
> And if you're not aware, source code is not restricted to tarballs. You
> can use git repos, svn and any mechanism the fetcher supports.(even local
> directories) The following URL lists the types the fetcher supports.
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
>
> The workflow now encouraged, is to use recipetool to assist in creating
> recipes. And then once the recipes are created, you can use devtool to
> actually do the iterative development on the application represented by the
> recipe.
>
> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>
>
>> Or perhaps this workflow is just not recommended or supported by the Yocto
>> Project? If that's the case, does Yocto only recommend building the
>> distribution then building the applications as completely separate
>> workflows? Or is there another workflow that I haven't stumbled across
>> yet?
>>
>>
> There is nothing "wrong" with building the image each time as a developer.
> However, it is time consuming to construct the full image and deploy it.
>
> A much more appealing mechanism would be to use devtool to build the
> recipe you are working on, and then use "devtool deploy-target" to then
> deploy the output to a live machine. This would require a writeable
> filesystem on the target device, but if that is available, the workflow and
> turnaround time is much faster.
>
>
>
>> Thanks again for all your help, and let me know if I can help clarify
>> anything,
>>
>>
>> Michael
>>
>>
>>
>>
>
[-- Attachment #2: Type: text/html, Size: 6595 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transitioning to Yocto, some basic questions about workflow
2015-11-11 15:36 ` Michael Habibi
@ 2015-11-11 18:04 ` Khem Raj
0 siblings, 0 replies; 7+ messages in thread
From: Khem Raj @ 2015-11-11 18:04 UTC (permalink / raw)
To: Michael Habibi; +Cc: yocto@yoctoproject.org
On Wed, Nov 11, 2015 at 7:36 AM, Michael Habibi <mikehabibi@gmail.com> wrote:
> Randy, I've had some time to think about your post and I had a follow up
> question. Let's say we had internal applications, you suggest that the
> fetcher can be used to pull down the git/svn repos that contains the
> internal application code. In this environment, where is the Yocto framework
> hosted? Is it also on a repo? Would then we have a Yocto repo for image
> building, and an application repo for application development? Is there a
> clean way of combining both Yocto and internal applications into a single
> repo, so our engineers simply have to do a 'git clone' and they'll have what
> they need to get started?
you will be better served if you prepare to deal with several git
repositories and use a sandboxing tool
like android repo or git submodules or may be shell scripts or
combo-layer tool. This can give you an easy way to clone your
project/workspace
There are a few ways you can go about your workflows it really depends
upon your internal team setup and nature of software you do
Lets say you have large teams of application software developers then
you might like to have a small platform/System-integration team
which is responsible for providing ADT to these teams and they can
work on applications independently. Meanwhile you can have this SI
team
to them help write and maintain recipes for these applications and
also maintain the platform builds which will include these
applications too
this way you can decouple the application development from platform
development and creation. ADT can hook into IDEs like eclipse and QT
Creator
if your dev community use these tools. You also dont have to train
large number of folks with OE/Yocto which for app developers could be
a time saver
Secondly, if you have large middleware and kernels and other parts of
platform that your developers touch then should be using the yocto
build env for full platform build, now there are addition productivity
tools like devtool which will help you develop a particular package
using internal SDKs under bitbake's control. Then you can use
recipetool to generate recipes for applications or bbappends and so
on. But its a new way of doing things so it will require training too.
see http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-modifying-source-code
for more on devtool.
>
> Thanks, still trying to wrap my head around this a bit.
>
> On Tue, Nov 10, 2015 at 6:06 PM, Randy Witt <randy.e.witt@linux.intel.com>
> wrote:
>>
>> Hi Michael,
>>
>> See my replies below.
>>
>> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>>
>>>
>>>
>>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>>> potential replacement distribution. However, I have a couple questions
>>> stemming from my research. We can leverage the existing Yocto build for
>>> various open source utilities where it pulls from the web, patches,
>>> builds,
>>> and installs into the deployable image. We would probably want to
>>> colocate
>>> the tarballs locally, because we would like to prevent people from having
>>> to fetch from the web during build cycles.
>>>
>>
>> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local
>> mirrors to try before using the upstream url in the recipe. So each person
>> building would at least have to fetch from the local mirror for a build, but
>> subsequent builds wouldn't require fetching again as long as the DL_DIR was
>> preserved.
>>
>> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
>>
>> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>>
>>>
>>> I see that Yocto thinks of the deployable image and applications as
>>> separate entities: first you'd build the distribution, then you would use
>>> ADT/cross-toolchain/etc to build the applications and install them
>>> separately. However, in our environment, we would need all of our custom
>>> IP
>>> applications to be built as part of the deployable image, and not as a
>>> separate procdure. For example, if I were to call "bitbake our-image", I
>>> would like the deployable image to contain 1) the kernel, 2) various core
>>> utilities and libraries for booting, and 3) our custom applications for
>>> our
>>> device.
>>>
>>>
>>> This means that somewhere in the yocto framework, we'd ideally have some
>>> source code somewhere that would also be compiled via recipes/classes
>>> that
>>> we'd have to custom write. Is there a best practice for this kind of
>>> workflow? I don't mind not having source checked in to our VCS for things
>>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a
>>> local
>>> server), but we likely wouldn't want to host tarballs for the
>>> applications
>>> we are writing and modifying day-to-day by dozens of engineers. Is there
>>> a
>>> place where this source would best fit? Would it be under build/tmp/etc,
>>> or
>>> perhaps we can locate the source under a layer directory, like
>>> meta/source/our-ip-applications?
>>>
>>
>> For all the internal applications at my previous employer we had a layer
>> for all internal items, it's quite common.
>>
>> And if you're not aware, source code is not restricted to tarballs. You
>> can use git repos, svn and any mechanism the fetcher supports.(even local
>> directories) The following URL lists the types the fetcher supports.
>>
>> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
>>
>> The workflow now encouraged, is to use recipetool to assist in creating
>> recipes. And then once the recipes are created, you can use devtool to
>> actually do the iterative development on the application represented by the
>> recipe.
>>
>> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>>
>>>
>>> Or perhaps this workflow is just not recommended or supported by the
>>> Yocto
>>> Project? If that's the case, does Yocto only recommend building the
>>> distribution then building the applications as completely separate
>>> workflows? Or is there another workflow that I haven't stumbled across
>>> yet?
>>>
>>
>> There is nothing "wrong" with building the image each time as a developer.
>> However, it is time consuming to construct the full image and deploy it.
>>
>> A much more appealing mechanism would be to use devtool to build the
>> recipe you are working on, and then use "devtool deploy-target" to then
>> deploy the output to a live machine. This would require a writeable
>> filesystem on the target device, but if that is available, the workflow and
>> turnaround time is much faster.
>>
>>
>>>
>>> Thanks again for all your help, and let me know if I can help clarify
>>> anything,
>>>
>>>
>>> Michael
>>>
>>>
>>>
>>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-11-11 18:05 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-11-10 22:27 Transitioning to Yocto, some basic questions about workflow Michael Habibi
2015-11-11 0:06 ` Randy Witt
2015-11-11 1:10 ` Michael Habibi
2015-11-11 2:33 ` Bob Cochran
2015-11-11 15:36 ` Michael Habibi
2015-11-11 18:04 ` Khem Raj
2015-11-11 14:00 ` Burton, Ross
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.