* parallel build of multiple machine/os? @ 2014-12-10 18:52 Koehler, Yannick (HP Networking) 2014-12-10 19:00 ` Khem Raj 0 siblings, 1 reply; 6+ messages in thread From: Koehler, Yannick (HP Networking) @ 2014-12-10 18:52 UTC (permalink / raw) To: yocto@yoctoproject.org Can Yocto build for example in a single bitbake call an image for x86, ppc, arm, others? Or is the call to bitbake limited to a single architecture? And if so, how? I see that lots of *-native package are built, and I guess is that for each machine/os those would be reused, as such, it appears it would be a benefit to be able to build multiple package version for different architecture/distro/machine in a single invocation of bitbake. -- Yannick Koehler ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: parallel build of multiple machine/os? 2014-12-10 18:52 parallel build of multiple machine/os? Koehler, Yannick (HP Networking) @ 2014-12-10 19:00 ` Khem Raj 2014-12-10 19:27 ` Maciek Borzecki 2014-12-11 6:55 ` Mike Looijmans 0 siblings, 2 replies; 6+ messages in thread From: Khem Raj @ 2014-12-10 19:00 UTC (permalink / raw) To: Koehler, Yannick (HP Networking); +Cc: yocto@yoctoproject.org > On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking) <yannick.koehler@hp.com> wrote: > > Can Yocto build for example in a single bitbake call an image for x86, ppc, arm, others? Or is the call to bitbake limited to a single architecture? And if so, how? > > I see that lots of *-native package are built, and I guess is that for each machine/os those would be reused, as such, it appears it would be a benefit to be able to build multiple package version for different architecture/distro/machine in a single invocation of bitbake. > parallel multi-machine builds have been discussed but not thought of worth doing. you could however share lot of common packages among machines sequentially. MACHINE=foo bitbake x MACHINE=bar bitbake y the second invocation would be able to share many artifacts from first builds e.g. native packages and nativesdk packages etc. so its generally faster than first build. > -- > Yannick Koehler > -- > _______________________________________________ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: parallel build of multiple machine/os? 2014-12-10 19:00 ` Khem Raj @ 2014-12-10 19:27 ` Maciek Borzecki 2014-12-11 6:55 ` Mike Looijmans 1 sibling, 0 replies; 6+ messages in thread From: Maciek Borzecki @ 2014-12-10 19:27 UTC (permalink / raw) To: Khem Raj; +Cc: yocto@yoctoproject.org, Koehler, Yannick (HP Networking) On śro, 2014-12-10 at 11:00 -0800, Khem Raj wrote: > > On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking) > <yannick.koehler@hp.com> wrote: > > Can Yocto build for example in a > single bitbake call an image for x86, ppc, arm, others? Or is the call > to bitbake limited to a single architecture? And if so, how? > > I see > that lots of *-native package are built, and I guess is that for each > machine/os those would be reused, as such, it appears it would be a > benefit to be able to build multiple package version for different > architecture/distro/machine in a single invocation of bitbake. > > > parallel multi-machine builds have been discussed but not thought of > worth doing. you could however share lot of common packages among > machines sequentially. Out of curiosity, are parallel builds using the same SSTATE_DIR and DL_DIR, but all other *_DIR variables distinct, considered safe? -- Maciej Borzęcki Senior Software Developer at Open-RnD Sp. z o.o., Poland www.open-rnd.pl mobile: +48 889 117 365, fax: +48 42 657 9079 Niniejsza wiadomość wraz z załącznikami może zawierać chronione prawem lub poufne informacje i została wysłana wyłącznie do wiadomości i użytku osób, do których została zaadresowana. Jeśli wiadomość została otrzymana przypadkowo zabrania się jej kopiowania lub rozsyłania do osób trzecich. W takim przypadku uprasza się o natychmiastowe zniszczenie wiadomości oraz poinformowanie nadawcy o zaistniałej sytuacji za pomocą wiadomości zwrotnej. Dziękujemy. This message, including any attachments hereto, may contain privileged or confidential information and is sent solely for the attention and use of the intended addressee(s). If you are not an intended addressee, you may neither use this message nor copy or deliver it to anyone. In such case, you should immediately destroy this message and kindly notify the sender by reply email. Thank you. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: parallel build of multiple machine/os? 2014-12-10 19:00 ` Khem Raj 2014-12-10 19:27 ` Maciek Borzecki @ 2014-12-11 6:55 ` Mike Looijmans 2014-12-11 19:42 ` Mark Hatle 1 sibling, 1 reply; 6+ messages in thread From: Mike Looijmans @ 2014-12-11 6:55 UTC (permalink / raw) To: yocto On 12/10/2014 08:00 PM, Khem Raj wrote: > >> On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking) <yannick.koehler@hp.com> wrote: >> >> Can Yocto build for example in a single bitbake call an image for x86, ppc, arm, others? Or is the call to bitbake limited to a single architecture? And if so, how? >> >> I see that lots of *-native package are built, and I guess is that for each machine/os those would be reused, as such, it appears it would be a benefit to be able to build multiple package version for different architecture/distro/machine in a single invocation of bitbake. > > parallel multi-machine builds have been discussed but not thought of worth doing. you could however share > lot of common packages among machines sequentially. > > [...] A more convincing use-case for multi-machine building is the emerging of "hybrids". The little/big architecture that puts different versions of ARM CPUs into a single machine is one example, the OMAPs with the DSP and PRUs are another. Boards with FPGAs can have several CPU architectures running on a single board. One could imagine that one would want to build the software for all these subsystems in a single call, and that one bitbake process would be managing that the build system gets properly used. Currently, these hybrids have to invoke bitbake for each subsystem, probably needlessly regenerating binaries, and once more for the one image to find them all and in darkness bind them... Met vriendelijke groet / kind regards, Mike Looijmans System Expert TOPIC Embedded Systems Eindhovenseweg 32-C, NL-5683 KH Best Postbus 440, NL-5680 AK Best Telefoon: (+31) (0) 499 33 69 79 Telefax: (+31) (0) 499 33 69 70 E-mail: mike.looijmans@topic.nl Website: www.topic.nl Please consider the environment before printing this e-mail Topic zoekt gedreven (embedded) software specialisten! http://topic.nl/vacatures/topic-zoekt-software-engineers/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: parallel build of multiple machine/os? 2014-12-11 6:55 ` Mike Looijmans @ 2014-12-11 19:42 ` Mark Hatle 2014-12-12 6:24 ` Mike Looijmans 0 siblings, 1 reply; 6+ messages in thread From: Mark Hatle @ 2014-12-11 19:42 UTC (permalink / raw) To: yocto On 12/11/14, 12:55 AM, Mike Looijmans wrote: > On 12/10/2014 08:00 PM, Khem Raj wrote: >> >>> On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking) <yannick.koehler@hp.com> wrote: >>> >>> Can Yocto build for example in a single bitbake call an image for x86, ppc, arm, others? Or is the call to bitbake limited to a single architecture? And if so, how? >>> >>> I see that lots of *-native package are built, and I guess is that for each machine/os those would be reused, as such, it appears it would be a benefit to be able to build multiple package version for different architecture/distro/machine in a single invocation of bitbake. >> >> parallel multi-machine builds have been discussed but not thought of worth doing. you could however share >> lot of common packages among machines sequentially. >> >> [...] > > A more convincing use-case for multi-machine building is the emerging of > "hybrids". The little/big architecture that puts different versions of ARM > CPUs into a single machine is one example, the OMAPs with the DSP and PRUs are > another. Boards with FPGAs can have several CPU architectures running on a > single board. One could imagine that one would want to build the software for > all these subsystems in a single call, and that one bitbake process would be > managing that the build system gets properly used. > > Currently, these hybrids have to invoke bitbake for each subsystem, probably > needlessly regenerating binaries, and once more for the one image to find them > all and in darkness bind them... > From a userspace perspective, this is more or less solved with the multilib configurations. Perhaps it would be possible to implement some kind of hybrid kernel configuration using the same type of multilib setup. (Note, this would be new work, it's not something we have today.) --Mark > Met vriendelijke groet / kind regards, > > Mike Looijmans > System Expert > > > TOPIC Embedded Systems > Eindhovenseweg 32-C, NL-5683 KH Best > Postbus 440, NL-5680 AK Best > Telefoon: (+31) (0) 499 33 69 79 > Telefax: (+31) (0) 499 33 69 70 > E-mail: mike.looijmans@topic.nl > Website: www.topic.nl > > Please consider the environment before printing this e-mail > > Topic zoekt gedreven (embedded) software specialisten! > http://topic.nl/vacatures/topic-zoekt-software-engineers/ > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: parallel build of multiple machine/os? 2014-12-11 19:42 ` Mark Hatle @ 2014-12-12 6:24 ` Mike Looijmans 0 siblings, 0 replies; 6+ messages in thread From: Mike Looijmans @ 2014-12-12 6:24 UTC (permalink / raw) To: yocto On 12/11/2014 08:42 PM, Mark Hatle wrote: > On 12/11/14, 12:55 AM, Mike Looijmans wrote: >> On 12/10/2014 08:00 PM, Khem Raj wrote: >>> >>>> On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking) >>>> <yannick.koehler@hp.com> wrote: >>>> >>>> Can Yocto build for example in a single bitbake call an image for x86, >>>> ppc, arm, others? Or is the call to bitbake limited to a single >>>> architecture? And if so, how? >>>> >>>> I see that lots of *-native package are built, and I guess is that for >>>> each machine/os those would be reused, as such, it appears it would be a >>>> benefit to be able to build multiple package version for different >>>> architecture/distro/machine in a single invocation of bitbake. >>> >>> parallel multi-machine builds have been discussed but not thought of worth >>> doing. you could however share >>> lot of common packages among machines sequentially. >>> >>> [...] >> >> A more convincing use-case for multi-machine building is the emerging of >> "hybrids". The little/big architecture that puts different versions of ARM >> CPUs into a single machine is one example, the OMAPs with the DSP and PRUs are >> another. Boards with FPGAs can have several CPU architectures running on a >> single board. One could imagine that one would want to build the software for >> all these subsystems in a single call, and that one bitbake process would be >> managing that the build system gets properly used. >> >> Currently, these hybrids have to invoke bitbake for each subsystem, probably >> needlessly regenerating binaries, and once more for the one image to find them >> all and in darkness bind them... >> > > From a userspace perspective, this is more or less solved with the multilib > configurations. Perhaps it would be possible to implement some kind of hybrid > kernel configuration using the same type of multilib setup. (Note, this would > be new work, it's not something we have today.) The OMAP PRU and DSP each run their own bootloader/kernels (the PRU is just bare metal code), and do not share code with the ARM that runs the rest. From the ARM's perspective, the DSP and PRU just need "firmware". If I put a Microblaze into the FPGA of a Zynq, I have a system that can run multiple kernels, on the built-in ARM CPUs and on the Microblaze(s). The ARM is then responsible for bootstrapping the microblaze, but it ends there, as the microblaze can run its own kernel and rootfs and applications, and they don't need to understand each other's binaries. One problem here is that the "firmware" for the microblaze would be a "image", and images and in particular the rootfs, for reasons yet unknown to me, always get rebuilt even if nothing changed and the output would be the same. I might succeed in spawning another bitbake from within the one that builds the ARM system, but on a 8-core system this sub-bitbake would also try to run 8 parallel processes, each potentially running 8 compiler threads. Some of these compilers eat RAM like crazy, so this is not going to be productive. It would be much better if the builds for the ARM and mircroblaze would share the load. Probably a very challenging thing to do currently. Met vriendelijke groet / kind regards, Mike Looijmans System Expert TOPIC Embedded Systems Eindhovenseweg 32-C, NL-5683 KH Best Postbus 440, NL-5680 AK Best Telefoon: (+31) (0) 499 33 69 79 Telefax: (+31) (0) 499 33 69 70 E-mail: mike.looijmans@topic.nl Website: www.topic.nl Please consider the environment before printing this e-mail Topic zoekt gedreven (embedded) software specialisten! http://topic.nl/vacatures/topic-zoekt-software-engineers/ ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-12-12 6:24 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-12-10 18:52 parallel build of multiple machine/os? Koehler, Yannick (HP Networking) 2014-12-10 19:00 ` Khem Raj 2014-12-10 19:27 ` Maciek Borzecki 2014-12-11 6:55 ` Mike Looijmans 2014-12-11 19:42 ` Mark Hatle 2014-12-12 6:24 ` Mike Looijmans
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.