* How should OE be used?
@ 2007-09-22 21:24 Philip Balister
2007-09-23 11:19 ` Leon Woestenberg
` (4 more replies)
0 siblings, 5 replies; 26+ messages in thread
From: Philip Balister @ 2007-09-22 21:24 UTC (permalink / raw)
To: openembedded-devel
[-- Attachment #1: Type: text/plain, Size: 2578 bytes --]
Recently, I've been spending some time thinking how people use OE for
building software. It seems like there are several groups of people
interested in and actually using OE for serious work (by serious I mean,
there is real money involved). At OEDEM, I would like to see some
discussion how the PE project can improve interaction with outside
developers and vendors.
As an example, how should OE interact with the company that sells
gumstix? Currently, they have a copy of buildroot they use to create
software for their product. The buildroot solution is not a perfect one
for them, see:
http://sourceforge.net/mailarchive/forum.php?thread_name=65AAF154-3C05-4836-952B-E5FF2E40E34C%40ecs.soton.ac.uk&forum_name=gumstix-users
Understanding that gumstix has a large customer base, how should they
use OE? I do not think a solution that has their customer base working
against .dev would be an improvement (stability wise) from their
"private" buildroot system. Also, the idea of adding that entire user
community to the oe-dev/user listservs would drown out other, equally
worthy lines of hardware.
Anyway, this is what has started me thinking. What I would like to do,
is create a list of how people use OE, and what concerns they have about
interactions with outside development teams. Once we have this list,
then we can have a conversation about what we can do to make things work
well for all OE developers and users.
Here are my preliminary thoughts:
How do people use OE?
* Basic install image and user adds packages
* Install image that user does not change, part of a "product"
Why is it important work effectively with third party developers
* Enhance projects (both OE and third parties)
* Create jobs for OE devs (that want them)
* Reduce friction
* Eliminate FUD
Basic issues for third party users of OE to build systems:
* What is OE? A distribution, or a distribution creation tool?
* Need to build product with a stable base.
* A way to keep proprietary content separate from OE.
* How do software developers use OE to write/debug code?
* What is the right way to support hardware?
Issues for OE developers:
* Third parties need to feed bug fixes and new work into .dev.
* How to not overwhelm OE devs with support requests.
* How does OE scale to large numbers of users?
For now, I'd like to focus on working out how people use OE and what
issues they have. Once we understand what problems exist, if any, we can
discuss solutions.
Philip
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3303 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-22 21:24 How should OE be used? Philip Balister
@ 2007-09-23 11:19 ` Leon Woestenberg
2007-09-23 12:33 ` Koen Kooi
2007-09-24 15:55 ` Tim Bird
2007-09-24 14:48 ` Darcy Watkins
` (3 subsequent siblings)
4 siblings, 2 replies; 26+ messages in thread
From: Leon Woestenberg @ 2007-09-23 11:19 UTC (permalink / raw)
To: openembedded-devel
Hello Philip, all,
On 9/22/07, Philip Balister <philip@balister.org> wrote:
> [...]
> For now, I'd like to focus on working out how people use OE and what
> issues they have. Once we understand what problems exist, if any, we can
> discuss solutions.
>
I became an OE user when I found that OE automated a lot more than
buildroot when I started adding packages. After that experience I
decided to use OE at work.
At work we create firmware, which is read-only embedded software which
the user cannot install to, or modify in any way (other than
overwrite) for 24/7 99.999% availability applications. Also, no build
dependency on target availability during build: No on-target
post-install scripts, the final image must be fully host-created. That
means, having full control of what happens, minimal cruft. Must build
on a multiple of (modern) Linux hosts distro's.
Bottom-line: paranoid control freak approach
I am the OE evangelist, that is, had to persuade to have my co-workers
see the benefits of using OE. That worked best when I showed them how
I could solve a problem more quickly than they could with buildroot,
or LFS. The problem often remains the steep learning curve, i.e.
people do often have some Makefile/autoconf experience but not
necessarely python/bitbake.
Bottom line: Show when OE works when the 'problem solving' leverage is highest.
How do we use OpenEmbedded:
- We cannot afford to follow the org.oe.dev. head developments. The
head is where fixes get fixed. Takes too much time away from the
developers.
- We cannot afford to follow head, but fix package versions. The OE
framework (bitbake, variables, behaviour) changes too much.
- We pick a fixed OE revision and stick with it during a project
(could be 2 years).
- I try to track OE head on several targets, to stay up-to-date when I
need to start a new project.
- Some of the packages have too much build and/or run-time
dependencies. Yes, both.
- We create a private overlay of minimal-configured packages and
discard redundant dependencies, we keep it in CVS (yes, I know...)
- We add our custom packages to our private overlay.
- I bring some new packages or fixes upstream, only for head.
- We do not use task-base approach, because it builds dependencies we
did not have before (a dependency regression).
- We have to keep good track of bitbake version, OE revision, private
overlay revision and host distro version, and know-good-combos'. We
need to be able fix a bug in the firmware in three years without any
regression. Virtual machines help.
Bottom line:
- OpenEmbedded works perfectly for our use, if we take very much care
in the way we use it. We might not do things the OE-way, simply we may
lack in-depth knowledge, or exposure to it.
Issues:
- Dependencies. The standard OE config seems targetted towards small
(handheld) computing devices, not necessarely with embedded software.
Everything with an installer is non-embedded software, IMHO.
-> I would like to see a OE group focus on minimal dependency distro.
I would join such a group, but cannot do that alone.
- Angstrom seems the de-facto distro OE is tested against, at least
upstream. Angstrom is excellent, but testing OE against Angstrom only
drives OE away from the embedded domain IMHO.
-> Unbitrot (Refresh) a distro other than Angstrom, preferably a more
minimal one.
- Documentation.
-> It's there, but you have to know what you are looking for. Most
people don't know, or don't look.
- I have yet to see a developer walk in with OE experience. Buildroot
knowledge is a big plus though.
The best thing I would envision (and I would propose as a OE vision),
is that OE would be funded by the semiconductor and board
manufacturers to support their silicon and boards,
so that in 3 years, everyone of them could mention "Enabled by OpenEmbedded".
My two cents,
Leon.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-23 11:19 ` Leon Woestenberg
@ 2007-09-23 12:33 ` Koen Kooi
2007-09-24 17:20 ` Tim Bird
2007-09-24 15:55 ` Tim Bird
1 sibling, 1 reply; 26+ messages in thread
From: Koen Kooi @ 2007-09-23 12:33 UTC (permalink / raw)
To: Using the OpenEmbedded metadata to build Distributions
Leon Woestenberg schreef:
> - We do not use task-base approach, because it builds dependencies we
> did not have before (a dependency regression).
Task-base is a 'works great for 95% of the users' kind of thing. It wasn't intended to be
the absolute minimum, which is why we have task-boot now and still need more tasks with
that purpose. Task-base will give you a starting point that is known to work (if not, file
bugs :)) from where you can slim it down or fatten it up. However, if you have 8mb or less
flash, you need something (vastly) different.
> -> I would like to see a OE group focus on minimal dependency distro.
> I would join such a group, but cannot do that alone.
You mean a minimal dependency image, a distro only sets policy, and angstrom has been
*extremely* carefull to not impose size decicions.
> - Angstrom seems the de-facto distro OE is tested against, at least
> upstream. Angstrom is excellent, but testing OE against Angstrom only
> drives OE away from the embedded domain IMHO.
It does not drive it away from the embedded domain, but it does make OE seem like a
monoculture. But that is not angstrom's or OE's fault. I think that's because it's pretty
much the only distro (not counting angstrom derivates like openmoko) that has active
developers *AND* is developed in .dev. If we need to checkout some svn/cvs/hg/git tree to
get your OE-based distro, you have already lost.
> -> Unbitrot (Refresh) a distro other than Angstrom, preferably a more
> minimal one.
Again, you mean images. I have an angstrom image in use that is below 900kB (jffs2) with
no postinsts and the only write action it 'needs' is the dropbear key, which can be
skipped if you really want that.
I and others are currently waiting for the nslu2 people to make up their mind about using
angstrom or not before we are going to work on proper embedded images and configurations
(e.g. a slimmer task-base / fatter task-boot, using busybox init and login, superstrip,
etc) for angstrom.
> - Documentation.
> -> It's there, but you have to know what you are looking for. Most
> people don't know, or don't look.
The canonical place is the OE usermanual, which is on the website and edited in mtn from
the org.openembedded.documentation branch.
regards,
Koen
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-22 21:24 How should OE be used? Philip Balister
2007-09-23 11:19 ` Leon Woestenberg
@ 2007-09-24 14:48 ` Darcy Watkins
2007-09-25 12:19 ` Stelios Koroneos
2007-09-24 16:14 ` Cliff Brake
` (2 subsequent siblings)
4 siblings, 1 reply; 26+ messages in thread
From: Darcy Watkins @ 2007-09-24 14:48 UTC (permalink / raw)
To: openembedded-devel
>How do people use OE?
>
> * Basic install image and user adds packages
> * Install image that user does not change, part of a "product"
A colleague of mine at a different site took an April snapshot of OE and
used this as the basis of a toolchain and linux distro for an embedded
product. They use the snapshot and have placed it into their own
version control system. This approach, although it gives them control
and stability, obviously isolates them from the upstream.
I tried using OE, but could not get it to complete a build for PowerPC
405 (neither uclibc or glibc - both failed but in different ways.
Uclibc gets cross compiler badness errors - I think it is part of this
powerpc vs ppc stuff in Linux). After days of frustration, I had to
bail out and use buildroot for this particular project instead.
Obviously OE has lots of ARM and x86 users, but the PPC support falls
behind because it isn't as popular. I'll check back for a later
project.
Nevertheless, our approach is:
- Need the host system to build the entire romable image
- Target does NOT support any form of package install scheme
- Upgrade is whole image rather than individual packages
- Config NV database is stored separate from the firmware image
(so that it doesn't get lost during upgrades)
- We do allow some flash to have a jffs2 filesystem for data
files and log storage as needed (not part of upgrade scheme).
>Why is it important work effectively with third party developers
>
> * Enhance projects (both OE and third parties)
> * Create jobs for OE devs (that want them)
> * Reduce friction
> * Eliminate FUD
I agree with the suggestion that corporate sponsored development should
primarily be driven by vendors (i.e. mainly the CPU vendors and board
vendors). This especially applies to chip families that don't enjoy as
wide a manufacturing base as ARM (e.g. Coldfire, PPC, AVR32).
I'd like to see a minimalist distro as an alternate to Angstrom, one
that has an objective to cleanly build for all supported chip families.
I think it is easier to get someone in my position to fix a port for an
add on package for my favorite CPU if I have a working basic system to
start with, but if I can't even build a basic system within a typical
evaluation timeframe, then I have no choice but to move on (in my case,
for the project at hand, I moved on to buildroot - but to be fair,
buildroot couldn't build its own linux kernel for PPC405 out of the box
either, even though I was able to get it to build the Linux / Xenomai
combo from DENX source RPMs).
I hope my frustrations and "giving up for now" is construed more as a
suggestion and challenge for the OE project team rather than as FUD.
:-)
Ultimately, once I get past the first hurdles, I'd like to have a distro
a bit more than minimalist, basically what it would take to implement
secure appliances and network infrastructure.
- Real time enhanced kernel
- SSL, SSH, login authentication better than the basic in busybox
- HTTPS, PHP5 and Python capable web server (e.g. appWeb)
- Client and server support for certificates
- Drivers for WiFi and WiMax
I have no need for graphics or any of that GUI jazz stuff (because I am
interested in infrastructure boxes not handheld devices), but I do see
need for some elements of web 2.0 technology, templates support and
internationalization / localization support (which is why I consider
PHP, python and SQLite support as important).
>Basic issues for third party users of OE to build systems:
>
> * What is OE? A distribution, or a distribution creation tool?
> * Need to build product with a stable base.
> * A way to keep proprietary content separate from OE.
> * How do software developers use OE to write/debug code?
> * What is the right way to support hardware?
I see two opportunities here.
1. OE could be a toolchain / distro maintenance tool for our products.
2. We could sell the HW for our products to OEMs in which case, OE
could be the basis of what we would want to provide to our customers.
(At this point, we would come under the category of a board vendor).
Proprietary content appears to be a fact of life, especially when it
comes to software defined radio. If you choose to have OE support
proprietary content, it becomes important to point out such dependencies
to the user while picking and choosing packages rather than having the
build fail because an attempt to retrieve a tarball gets an
access-denied. OE could provide a framework or even just a
philosophical approach to handle proprietary add-ons - but leave the
implementation to the vendors who will profit from them.
I see proprietary add-ons mainly being in the area of support for
specific types of hardware. In an almost ideal world, we would see
proprietary add on drivers being temporary from the point a vendor
introduces a new device until the time they feel that it is safe to
release the driver code as FOSS (i.e. once the market edge has been
realized and they don't fear that it would give instant gratification to
a competitor). Wireless hardware support is an area where you could see
lots of this happening.
>Issues for OE developers:
>
> * Third parties need to feed bug fixes and new work into .dev.
> * How to not overwhelm OE devs with support requests.
> * How does OE scale to large numbers of users?
Perhaps the project needs to be partitioned into sub-projects for:
- toolchain and framework
- minimalistic systems for all supported platforms
- distributions (each one a project in its own right)
- some major areas of related packages (e.g. graphics/GUI, wireless,
security, etc) likely to be included all-or-none in a distro.
Also, from the point of view of someone who may want to market products
based on an OE toolchain and distro, it is not acceptable to have OE be
broken. There needs to be a QA process that focuses the project into a
development phase, then a fix and cleanup phase, then a release and
maintenance phase. There would need to be multiple branches (e.g.
org.oe.dev, org.oe.beta.v3, org.oe.stable.v2, org.oe.stable.v1).
Someone should be able to lock into a stable branch with some sort of
expectation that it will work (at least as documented - i.e. cases of
known issues to be fixed only in a next version is a fact of life).
They also need to be able to trust that updates to the stable branches
will be according to a conservative maintenance policy. Otherwise they
will just implement their own branch (which could be an approach in
itself if the OE project chooses to be concerned only with development
and leave QA / maintenance as a value-add service for downstream vendors
to provide and even charge for).
Regards,
Darcy
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-23 11:19 ` Leon Woestenberg
2007-09-23 12:33 ` Koen Kooi
@ 2007-09-24 15:55 ` Tim Bird
2007-09-24 16:35 ` Philip Balister
2007-09-24 18:04 ` Leon Woestenberg
1 sibling, 2 replies; 26+ messages in thread
From: Tim Bird @ 2007-09-24 15:55 UTC (permalink / raw)
To: Leon Woestenberg; +Cc: openembedded-devel
Leon Woestenberg wrote:
> I became an OE user when I found that OE automated a lot more than
> buildroot when I started adding packages. After that experience I
> decided to use OE at work.
...
Leon,
I found the description of your experiences with OpenEmbedded
and your list of issues to be very valuable.
I have added your message text to the elinux wiki at:
http://elinux.org/Open_Embedded
I hope this is OK with you. Please let me know if there is any
problem with this.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-22 21:24 How should OE be used? Philip Balister
2007-09-23 11:19 ` Leon Woestenberg
2007-09-24 14:48 ` Darcy Watkins
@ 2007-09-24 16:14 ` Cliff Brake
2007-09-24 18:36 ` Craig Hughes
2007-09-24 18:48 ` Lorn Potter
4 siblings, 0 replies; 26+ messages in thread
From: Cliff Brake @ 2007-09-24 16:14 UTC (permalink / raw)
To: openembedded-devel
On 9/22/07, Philip Balister <philip@balister.org> wrote:
> For now, I'd like to focus on working out how people use OE and what
> issues they have. Once we understand what problems exist, if any, we can
> discuss solutions.
I use OE in much the same way as Leon. I find a OE/Bitbake snapshot
that is stable and lock it down for project work. I generally don't
upgrade until I need a new feature, or I have some time to work
through issues. I have several dev boards that I use to track the
HEAD that I use for contributing to OE, and keeping up with what is
going on.
For me, this works fine, but then I keep up with what is going on, so
I have a pretty good feel for when it is a good time to upgrade
projects, etc. The occasional OE user would probably have a much more
difficult time. I'm not sure a stable branch would help a lot, as it
would consume a lot of developer time that is probably better spent
moving things forward Perhaps something more like kernel release
process would work where we shoot for stable releases (perhaps
quarterly) where the focus right before the release is to get things
stable for a number of architectures. But, then I'm not sure there is
a big advantage of this over just doing more/better autobuilds and
testing.
Cliff
--
=======================
Cliff Brake
http://bec-systems.com
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 15:55 ` Tim Bird
@ 2007-09-24 16:35 ` Philip Balister
2007-09-24 17:14 ` Tim Bird
2007-09-24 18:04 ` Leon Woestenberg
1 sibling, 1 reply; 26+ messages in thread
From: Philip Balister @ 2007-09-24 16:35 UTC (permalink / raw)
To: openembedded-devel; +Cc: Leon Woestenberg
[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]
Tim,
Hopefully, we can address some of Leon's and other's issues during OEDEM
in two weeks. So be prepared to update the elinux.org web page :)
Philip
Tim Bird wrote:
> Leon Woestenberg wrote:
>> I became an OE user when I found that OE automated a lot more than
>> buildroot when I started adding packages. After that experience I
>> decided to use OE at work.
> ...
>
> Leon,
>
> I found the description of your experiences with OpenEmbedded
> and your list of issues to be very valuable.
>
> I have added your message text to the elinux wiki at:
> http://elinux.org/Open_Embedded
>
> I hope this is OK with you. Please let me know if there is any
> problem with this.
>
> -- Tim
>
> =============================
> Tim Bird
> Architecture Group Chair, CE Linux Forum
> Senior Staff Engineer, Sony Corporation of America
> =============================
>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3303 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 16:35 ` Philip Balister
@ 2007-09-24 17:14 ` Tim Bird
0 siblings, 0 replies; 26+ messages in thread
From: Tim Bird @ 2007-09-24 17:14 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel, Leon Woestenberg, Matthew Locke
Philip Balister wrote:
> Tim,
>
> Hopefully, we can address some of Leon's and other's issues during OEDEM
> in two weeks. So be prepared to update the elinux.org web page :)
I'll be glad to. :-)
BTW - I still have some unspent money from CELF budgeted for OE
improvements (not a lot, but some). Also, I have a machine dedicated
in the CELF test lab for OE work. Last year, Marcin did some contract
work for us. I don't think we're leveraging what he did very well.
In terms of what CELF is interested in going forward: we would very
much like to see well-maintained minimal distribution that works
on several architectures. I can use some of the above
resources for this work.
I don't want to discuss numbers on an open list, but details
are available upon request.
Can you please factor this into the OEDEM discussions?
I won't be at OEDEM, but I'll be at ELCE in Austria, November 2-3.
(see http://www.celinux.org/elc_europe07/elc_europe_index.html)
Matt Locke will be giving a talk on Open Embedded there. I'm not
sure where the main OE developers are located in Europe, but if
it's convenient, it would be nice to meet there and discuss what
CELF can do to help out.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-23 12:33 ` Koen Kooi
@ 2007-09-24 17:20 ` Tim Bird
2007-09-24 17:43 ` Koen Kooi
0 siblings, 1 reply; 26+ messages in thread
From: Tim Bird @ 2007-09-24 17:20 UTC (permalink / raw)
To: openembedded-devel
Koen Kooi wrote:
> If we need to checkout some svn/cvs/hg/git tree to
> get your OE-based distro, you have already lost.
I strongly agree with this sentiment. I would like to see something
even further. I would like to see downloadable minimal distro *binaries*
(not just source) for all major architectures.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 17:20 ` Tim Bird
@ 2007-09-24 17:43 ` Koen Kooi
2007-09-24 18:01 ` Tim Bird
0 siblings, 1 reply; 26+ messages in thread
From: Koen Kooi @ 2007-09-24 17:43 UTC (permalink / raw)
To: openembedded-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Tim Bird schreef:
> Koen Kooi wrote:
>> If we need to checkout some svn/cvs/hg/git tree to
>> get your OE-based distro, you have already lost.
>
> I strongly agree with this sentiment. I would like to see something
> even further. I would like to see downloadable minimal distro *binaries*
> (not just source) for all major architectures.
Like http://www.angstrom-distribution.org/unstable/images/? That has armv4-oabi, armv4t,
armv5te, armv5teb, armv6, ppc603, i486, i586, i686 and soon ppc405 and avr32. A uclibc
variant is in the works as well :)
regards,
Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFG9/cnMkyGM64RGpERAonNAJ0VzleL5gJ2gOdxRF0w43fYzErYdwCdEKwK
36NC+lf72JouvWSMQ1kqG/k=
=+cYC
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 17:43 ` Koen Kooi
@ 2007-09-24 18:01 ` Tim Bird
0 siblings, 0 replies; 26+ messages in thread
From: Tim Bird @ 2007-09-24 18:01 UTC (permalink / raw)
To: openembedded-devel
Koen Kooi wrote:
> Tim Bird schreef:
>>> Koen Kooi wrote:
>>>> If we need to checkout some svn/cvs/hg/git tree to
>>>> get your OE-based distro, you have already lost.
>>> I strongly agree with this sentiment. I would like to see something
>>> even further. I would like to see downloadable minimal distro *binaries*
>>> (not just source) for all major architectures.
>
> Like http://www.angstrom-distribution.org/unstable/images/? That has armv4-oabi, armv4t,
> armv5te, armv5teb, armv6, ppc603, i486, i586, i686 and soon ppc405 and avr32. A uclibc
> variant is in the works as well :)
At a first glance, this is quite possibly exactly what I want.
Thanks!!
I'll look further and see if this will meet my needs.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 15:55 ` Tim Bird
2007-09-24 16:35 ` Philip Balister
@ 2007-09-24 18:04 ` Leon Woestenberg
2007-09-24 19:23 ` Koen Kooi
1 sibling, 1 reply; 26+ messages in thread
From: Leon Woestenberg @ 2007-09-24 18:04 UTC (permalink / raw)
To: openembedded-devel, Tim Bird
Hello Tim,
On 9/24/07, Tim Bird <tim.bird@am.sony.com> wrote:
> I found the description of your experiences with OpenEmbedded
> [...]
> I have added your message text to the elinux wiki at:
> http://elinux.org/Open_Embedded
>
> I hope this is OK with you. [...]
>
This is fine with me.
You may also want to add Cliff Brake's comments, or Darcy's comments.
I think our comments are very much alike, but I hope more OE users
will respond to Philip's question. (Please do so if you read this) so
that we can address improvements.
From Koen's response, I read that some of the "issues" I mention are
already solved in some way. This then proves that the power of OE
sometimes is too hidden for the beginning and even intermediate user,
which is an issue by itself.
I think (feel) that OE target development is driven by two main drivers:
- target board availability for the enthousiasts (give free boards
away to them!)
- support for a certain target architecture mixed with the automation
leverage of OE. We mainly need arch and toolchain expertise here. If
that's in place, the rest is easy.
I'm still thinking about how any funding could be spent wisely, but I
would like to sleep a night on that.
By the way, I'm not sure if I can attend OEDEM myself (double booked),
but I try to be at ELCE and the 9th Real-Time Linux Workshop (RTLWS)
which runs in parallel.
I think the latter event is interesting for OE developers as well.
Best regards,
--
Leon
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-22 21:24 How should OE be used? Philip Balister
` (2 preceding siblings ...)
2007-09-24 16:14 ` Cliff Brake
@ 2007-09-24 18:36 ` Craig Hughes
2007-09-24 19:27 ` Philip Balister
2007-09-24 21:37 ` Richard Purdie
2007-09-24 18:48 ` Lorn Potter
4 siblings, 2 replies; 26+ messages in thread
From: Craig Hughes @ 2007-09-24 18:36 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
On Sep 22, 2007, at 2:24 PM, Philip Balister wrote:
> Recently, I've been spending some time thinking how people use OE
> for building software. It seems like there are several groups of
> people interested in and actually using OE for serious work (by
> serious I mean, there is real money involved). At OEDEM, I would
> like to see some discussion how the PE project can improve
> interaction with outside developers and vendors.
>
> As an example, how should OE interact with the company that sells
> gumstix? Currently, they have a copy of buildroot they use to
> create software for their product. The buildroot solution is not a
> perfect one for them, see:
>
> http://sourceforge.net/mailarchive/forum.php?
> thread_name=65AAF154-3C05-4836-952B-E5FF2E40E34C%
> 40ecs.soton.ac.uk&forum_name=gumstix-users
>
> Understanding that gumstix has a large customer base, how should
> they use OE? I do not think a solution that has their customer base
> working against .dev would be an improvement (stability wise) from
> their "private" buildroot system. Also, the idea of adding that
> entire user community to the oe-dev/user listservs would drown out
> other, equally worthy lines of hardware.
My current plan is to do something similar (but more tightly meshed
with upstream) to what we're doing with buildroot. Actually, the
buildroot we use has been more-or-less forked from the main one with
not a lot of back-and-forth between ours and upstream for a while now
because they've drifted so much over time. My plan more specifically
with OE is to create my own monotone repo which we'll point gumstix
users/developer at. We'll have a "org.gumstix.oe" branch in there
which will be the recommended branch for gumstix folks to pull from.
I (or anyone who wants to live through the pain) can always pull/
propagate/merge from org.openembedded.dev, and I'll do so over time
into say a org.gumstix.dev; when stuff is stable enough, I'll then
propagate it over to org.gumstix.oe which the users will then get.
In theory, monotone will make this multi-branch, multi-repository
approach somewhat manageable. One downside with monotone appears to
be that as the revision DB grows, it's really pretty painfully slow.
Hopefully that won't get a lot worse if there are lots of these inter-
branch updates going on.
I then also plan to set up a .ipkg repo with tons of stuff pre-built
so folks can just download/install the binaries and not have to mess
with compiling themselves at all, which is a big win vs buildroot.
> Anyway, this is what has started me thinking. What I would like to
> do, is create a list of how people use OE, and what concerns they
> have about interactions with outside development teams. Once we
> have this list, then we can have a conversation about what we can
> do to make things work well for all OE developers and users.
The one main high-level issue I've had so far with OE is the notion
of "machine" vs "distro" vs "image", which doesn't cleanly overlap
with the existing concepts we use for gumstix. For gumstix, a
machine is a motherboard (with varying amounts of flash storage) +
some combo of daughtercards (not a fixed list of features). Distro
kinds of makes sense. "image" kind of makes sense too, but has some
problems where package selection overlaps with the concepts of
"machine" which don't quite make sense in the gumstix situation. I
don't think these problems are insoluble though, and partly they're
I'm sure just a matter of my own misunderstanding and lack of deep-
sightedness on the situation. For example, different "flavors" of
gumstix verdex have different amounts of flash on them. I'd like to
have one "type this and it'll work" command for gumstix users where
they can answer a minimal number of questions like "which
motherboard?", "how much flash?", and "which daughtercards?" and then
it'll go ahead and figure out which machine/image to build.
> Here are my preliminary thoughts:
>
> How do people use OE?
>
> * Basic install image and user adds packages
Yes, our "factory image" will be OE generated. We'll have a ipkg
archive online for folks to be able to just grab and install pkgs.
> * Install image that user does not change, part of a "product"
Yes. Our factory image will work well enough for many people that
they won't need extra pkgs.
> Why is it important work effectively with third party developers
>
> * Enhance projects (both OE and third parties)
> * Create jobs for OE devs (that want them)
Yeah, good luck with that :) We're a poor, low-margin hardware
company ;P I wish we paid *ourselves*....
> * Reduce friction
> * Eliminate FUD
>
> Basic issues for third party users of OE to build systems:
>
> * What is OE? A distribution, or a distribution creation tool?
I'm 95% certain at this point that I'm going to be building a
"gumstix distro" on top of angstrom, ie import angstrom and modify
some stuff to get a bit tighter on the specific hardware/software we
want included/left out. So it's kind of both a distro and a distro
creation tool :)
> * Need to build product with a stable base.
Definitely. I think the way to do that cleanly-ish is to use the
multi-repo/multi-branch features of monotone effectively.
> * A way to keep proprietary content separate from OE.
Not a problem for us; we have nothing "proprietary". Every line of
code we write is "open". There will always be content which isn't
technically "proprietary" but which makes no sense outside of a
gumstix (like say, our fbcon splash logo or something, or some of our
default config files) which would fall in this category though probably.
> * How do software developers use OE to write/debug code?
Right now actually, this part's pretty painful. Figuring out a nice
way to have the .bb patch-applicator interact nicely with quilt
series files instead of having to copy/paste/sed~^~file://~;~
$~;patch=1~ is a real PITA. I had a similar issue with buildroot
which I basically fixed on the packages I care about by replacing
their patch-applicator script with a call to quilt to apply my series
files instead. Then doing actual development is just a matter of
cd'ing to the build directory, making changes using quilt, and then
the build scripts are automagically updated via the magic of quilt.
I haven't yet taken a look at what'd be involved in doing that within
OE, but if anyone's tried, or looked at it, or has any pointers on
where to start looking, lmk.
> * What is the right way to support hardware?
The right way is y'all write all my code for me (for free, remember),
and I go on vacation to Hawaii for a few months :)
> Issues for OE developers:
>
> * Third parties need to feed bug fixes and new work into .dev.
An early issue I've had (partly due to my not understanding OE before
starting to whack away at some code) is rapid drift between my
codebase, my submitted patches, and what actually gets into .dev
eventually. This leads to a propagate/merge nightmare which is not
made much easier by OE's habit of not telling you which file it's
showing you the 3-way diff for if you're not working in a GUI
enviroment but it's just launching vi to do the diff3's. That
probably just means I need to get used to running X all the time and
using some graphical diff tool, but it's still annoying that the
filenames are in a separate window from where the diff's showing.
> * How to not overwhelm OE devs with support requests.
In the gumstix case, I'm thinking that the best way of dealing with
this is to basically buffer .dev with org.gumstix.oe and
org.gumstix.dev -- our users will "not necessarily know" that
they're using OE. Our support wiki etc will mention that OE is under
the hood, but it'll try and divert all support stuff to us first, not
to openembedded.org; the danger here is that OE doesn't know what's
going on with a big chunk of its (indirect) users, and pressure gets
applied to effectively fork (this is what's basically happened with
buildroot for us). On the other hand, you guys really don't want
7,000 gumstix users (some of whom have @aol.com email addresses and
stuff) to start spamming the OE dev lists and bugtracker.
> * How does OE scale to large numbers of users?
Well, I think there are going to be 2 questions here actually: How
does OE scale, and how well does monotone scale. I'm not sure
monotone's ever had a project as large as OE with as many users pull/
sync'ing as it might face with the OE+gumstix combination of size +
developer base. Evidence so far shows that monotone might have some
serious issues. The concurrency-of-access one is the most
noticeable, given how slow monotone repo ops are. If one person is
pull'ing and it takes 5 minutes to do so, and locks the DB so nobody
else can pull at the same time, that's really really not going to
work. From my subversion logs for the gumstix buildroot, we get as
many as 400-600 "svn update"/"svn checkout" operations per day, most
of which are during office hours in the US. The most-hit hours
actually have over 100 in a single hour. The most-hit minutes
frequently get dozens and sometimes (rarely) even hundreds. If those
ops become mutually locked-out and take 5-10 minutes each now, it's
going to seriously piss off quite a few of our customers/users. Some
of that traffic appears to be messed-up scripts trying to pull the
whole repo lots of times in a single minute, but we actually deal
with that not too badly today. Getting DoS'd by those badly written
scripts would be not good.
C
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-22 21:24 How should OE be used? Philip Balister
` (3 preceding siblings ...)
2007-09-24 18:36 ` Craig Hughes
@ 2007-09-24 18:48 ` Lorn Potter
2007-09-24 19:01 ` Craig Hughes
2007-09-24 20:52 ` Richard Purdie
4 siblings, 2 replies; 26+ messages in thread
From: Lorn Potter @ 2007-09-24 18:48 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
I have always thought that oe needed an easier way to choose packages,
like buildroot.
Some gui that can edit all the conf's and task's, and image.bb's
painlessly and easily would make a tom of difference.
The learning curve for oe is too high.
Take buildroot, for example. A person can download it, run make
menuconfig, set up the system, arch, and select packages, all in one go.
You do not need to know that you should have to either edit or create
your own tasks, or image .bb's, much less the syntax for those files.
--
Lorn 'ljp' Potter
Software Engineer, Systems Group, MES, Trolltech
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 18:48 ` Lorn Potter
@ 2007-09-24 19:01 ` Craig Hughes
2007-09-24 19:39 ` Lorn Potter
2007-09-24 20:52 ` Richard Purdie
1 sibling, 1 reply; 26+ messages in thread
From: Craig Hughes @ 2007-09-24 19:01 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
On Sep 24, 2007, at 11:48 AM, Lorn Potter wrote:
> I have always thought that oe needed an easier way to choose packages,
> like buildroot.
>
> Some gui that can edit all the conf's and task's, and image.bb's
> painlessly and easily would make a tom of difference.
> The learning curve for oe is too high.
>
> Take buildroot, for example. A person can download it, run make
> menuconfig, set up the system, arch, and select packages, all in
> one go.
> You do not need to know that you should have to either edit or create
> your own tasks, or image .bb's, much less the syntax for those files.
You're kind of comparing apples to tangerines though. In buildroot,
yes, for packages which already exist (and work properly -- there's
much dead wood in buildroot) you can select them to be compiled using
a menuconfig/xconfig thing. But in OE, you could just download and
install the .ipkg instead, which is a lot cleaner. Buildroot is
missing 90% of the packages which are in OE, and for those, in
buildroot, you'd have to write your own package/*.mk, as well as
figure out what patches you need to implement to get the package to
cross-compile, etc. And you definitely need to learn the ins and
outs of how buildroot does things internally to implement one of
those *.mk files correctly.
But your basic premise of select-packages-to-go-into-image using some
kind of UI thing does make sense. I just don't think it's as bad in
OE as you think, nor as good in buildroot ;)
C
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 18:04 ` Leon Woestenberg
@ 2007-09-24 19:23 ` Koen Kooi
0 siblings, 0 replies; 26+ messages in thread
From: Koen Kooi @ 2007-09-24 19:23 UTC (permalink / raw)
To: Using the OpenEmbedded metadata to build Distributions
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Leon Woestenberg schreef:
> I think (feel) that OE target development is driven by two main drivers:
> - target board availability for the enthousiasts (give free boards
> away to them!)
And PSUs for the boards! I'm running low on 12V ones ;)
> - support for a certain target architecture mixed with the automation
> leverage of OE. We mainly need arch and toolchain expertise here. If
> that's in place, the rest is easy.
>
> I'm still thinking about how any funding could be spent wisely, but I
> would like to sleep a night on that.
On top of my wishlist:
Having (or access to) an autobuilder with enough diskspace to do multimachine build for
glibc, eglibc and uclibc for all machines in OE (About 100GB peak when using rm_work)
would be great. It would need >=1GB ram, access to a source mirror and have plenty diskspace.
regards,
Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFG+A7OMkyGM64RGpERAk3pAJ9l6GLdSpQFTNhSZF0SJY9SOUBnUQCgtj1u
gW/WzvsTu35j8SZ3imxHFFg=
=r/3/
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 18:36 ` Craig Hughes
@ 2007-09-24 19:27 ` Philip Balister
2007-09-24 21:37 ` Richard Purdie
1 sibling, 0 replies; 26+ messages in thread
From: Philip Balister @ 2007-09-24 19:27 UTC (permalink / raw)
To: openembedded-devel
[-- Attachment #1: Type: text/plain, Size: 648 bytes --]
Craig Hughes wrote:
>> * Create jobs for OE devs (that want them)
>
> Yeah, good luck with that :) We're a poor, low-margin hardware
> company ;P I wish we paid *ourselves*....
Hopefully, we can get paid by your customers to do things that cause
them to buy more gumstix ....
I have the netwifimicroSD board right? We have a couple of days for
hacking around scheduled for OEDEM, I'm going to try and get support for
the entire system in OE that weekend. Unless we take of and drink beer :)
If I can this thesis off my back, I will try and get this hw in the
trial run of the Maniac challenge in late October.
Philip
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3303 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 19:01 ` Craig Hughes
@ 2007-09-24 19:39 ` Lorn Potter
2007-09-24 20:33 ` Craig Hughes
0 siblings, 1 reply; 26+ messages in thread
From: Lorn Potter @ 2007-09-24 19:39 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
Craig Hughes wrote:
> On Sep 24, 2007, at 11:48 AM, Lorn Potter wrote:
>
>> I have always thought that oe needed an easier way to choose packages,
>> like buildroot.
>>
>> Some gui that can edit all the conf's and task's, and image.bb's
>> painlessly and easily would make a tom of difference.
>> The learning curve for oe is too high.
>>
>> Take buildroot, for example. A person can download it, run make
>> menuconfig, set up the system, arch, and select packages, all in
>> one go.
>> You do not need to know that you should have to either edit or create
>> your own tasks, or image .bb's, much less the syntax for those files.
>
> You're kind of comparing apples to tangerines though.
not really,.
> In buildroot,
> yes, for packages which already exist (and work properly -- there's
> much dead wood in buildroot) you can select them to be compiled using
> a menuconfig/xconfig thing. But in OE, you could just download and
> install the .ipkg instead, which is a lot cleaner.
Not necessarily. Where's the ipkg for a custom device that hasn't been
built yet?
> Buildroot is
> missing 90% of the packages which are in OE, and for those, in
Most people are not going to want to build 90% of the packages in oe,
especially for embedded/small devices.
We have a custom build script for the greenphone that builds the
toolchain, uses those libs/binutils for the system, then
downloads/builds 24 packages. at _most_.
> buildroot, you'd have to write your own package/*.mk, as well as
> figure out what patches you need to implement to get the package to
> cross-compile, etc. And you definitely need to learn the ins and
> outs of how buildroot does things internally to implement one of
> those *.mk files correctly.
Makefile format is more widely known than the format and syntax used in OE.
>
> But your basic premise of select-packages-to-go-into-image using some
> kind of UI thing does make sense. I just don't think it's as bad in
> OE as you think, nor as good in buildroot ;)
--
Lorn 'ljp' Potter
Software Engineer, Systems Group, MES, Trolltech
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 19:39 ` Lorn Potter
@ 2007-09-24 20:33 ` Craig Hughes
0 siblings, 0 replies; 26+ messages in thread
From: Craig Hughes @ 2007-09-24 20:33 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
On Sep 24, 2007, at 12:39 PM, Lorn Potter wrote:
> Not necessarily. Where's the ipkg for a custom device that hasn't been
> built yet?
> Most people are not going to want to build 90% of the packages in oe,
> especially for embedded/small devices.
> Makefile format is more widely known than the format and syntax
> used in OE.
All good points.
C
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 18:48 ` Lorn Potter
2007-09-24 19:01 ` Craig Hughes
@ 2007-09-24 20:52 ` Richard Purdie
2007-09-24 21:11 ` Lorn Potter
1 sibling, 1 reply; 26+ messages in thread
From: Richard Purdie @ 2007-09-24 20:52 UTC (permalink / raw)
To: openembedded-devel
On Tue, 2007-09-25 at 04:48 +1000, Lorn Potter wrote:
> I have always thought that oe needed an easier way to choose packages,
> like buildroot.
>
> Some gui that can edit all the conf's and task's, and image.bb's
> painlessly and easily would make a tom of difference.
> The learning curve for oe is too high.
This is something I plan to raise at OEDEM.
It should be relatively well known by now that I want to improve bitbake
to the extent where writing some kind of UI around it is straightforward
and we can start improving the user experience from this angle.
Bitbake trunk now has some foundations for this in place although there
are some gaps and a few regressions that need fixing. There is no new UI
itself yet.
The job isn't actually that difficult, the problem is that the learning
curve anyone implementing it faces with no previous experience is
steep.
Personally, the big problem I have is the lack of time to do it and I
suspect others with the experience also have time issues :-(.
> Take buildroot, for example. A person can download it, run make
> menuconfig, set up the system, arch, and select packages, all in one
> go.
> You do not need to know that you should have to either edit or create
> your own tasks, or image .bb's, much less the syntax for those files.
Playing devil's advocate, take Poky, for example. A person can
download/check it out, edit one simple configuration file
(comment/uncomment lines), source a file and go :).
So it is possible.
Cheers,
Richard
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 20:52 ` Richard Purdie
@ 2007-09-24 21:11 ` Lorn Potter
2007-09-24 22:35 ` Richard Purdie
0 siblings, 1 reply; 26+ messages in thread
From: Lorn Potter @ 2007-09-24 21:11 UTC (permalink / raw)
To: openembedded-devel; +Cc: openembedded-devel
Richard Purdie wrote:
> On Tue, 2007-09-25 at 04:48 +1000, Lorn Potter wrote:
>> I have always thought that oe needed an easier way to choose packages,
>> like buildroot.
>>
>> Some gui that can edit all the conf's and task's, and image.bb's
>> painlessly and easily would make a tom of difference.
>> The learning curve for oe is too high.
>
> This is something I plan to raise at OEDEM.
>
> It should be relatively well known by now that I want to improve bitbake
> to the extent where writing some kind of UI around it is straightforward
> and we can start improving the user experience from this angle.
Excellent.
>> Take buildroot, for example. A person can download it, run make
>> menuconfig, set up the system, arch, and select packages, all in one
>> go.
>> You do not need to know that you should have to either edit or create
>> your own tasks, or image .bb's, much less the syntax for those files.
>
> Playing devil's advocate, take Poky, for example. A person can
> download/check it out, edit one simple configuration file
> (comment/uncomment lines), source a file and go :).
This is ok for an individual with a known device, but for a company that
needs to produce their own distribution, for a new device, Poky is not a
choice. _These_ are the people/companies that the OE/Foundation should
also be targeting to grow.
--
Lorn 'ljp' Potter
Software Engineer, Systems Group, MES, Trolltech
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 18:36 ` Craig Hughes
2007-09-24 19:27 ` Philip Balister
@ 2007-09-24 21:37 ` Richard Purdie
1 sibling, 0 replies; 26+ messages in thread
From: Richard Purdie @ 2007-09-24 21:37 UTC (permalink / raw)
To: openembedded-devel
On Mon, 2007-09-24 at 11:36 -0700, Craig Hughes wrote:
> The one main high-level issue I've had so far with OE is the notion
> of "machine" vs "distro" vs "image", which doesn't cleanly overlap
> with the existing concepts we use for gumstix. For gumstix, a
> machine is a motherboard (with varying amounts of flash storage) +
> some combo of daughtercards (not a fixed list of features). Distro
> kinds of makes sense. "image" kind of makes sense too, but has some
> problems where package selection overlaps with the concepts of
> "machine" which don't quite make sense in the gumstix situation. I
> don't think these problems are insoluble though, and partly they're
> I'm sure just a matter of my own misunderstanding and lack of deep-
> sightedness on the situation. For example, different "flavors" of
> gumstix verdex have different amounts of flash on them. I'd like to
> have one "type this and it'll work" command for gumstix users where
> they can answer a minimal number of questions like "which
> motherboard?", "how much flash?", and "which daughtercards?" and then
> it'll go ahead and figure out which machine/image to build.
I suspect you just need to think more along the way OE works and this
will start to become clearer. If you look at poky's local.conf.sample
file, its aimed to give the user an "edit this single file and choose
between the significant bits of Poky's functionality" experience which
is similar to what you mention above.
Once you have a single conf file controlling what you need if you really
want you can write a script that generates it in a nice ncurses
interface but I don't think you have to do that.
To breakdown where I'd personally put the lines in your case, I'd have a
machine representing the motherboard (you have two, pxa255 and 270
iirc). That machine.conf file would default to the maximal configuration
of the motherboard i.e. including all drivers. You would then be able to
override the options. You could have "sub-machine" files with some
preset combinations too I guess. So the layout becomes:
gumstix-pxa270.conf:
GUMSTIX_MACHINE_FEATURES ?= "ethernet screen wifi bluetooth"
MACHINE_FEATURES = "${GUMSTIX_MACHINE_FEATURES}"
gumstix-pxa270-wifi.conf:
MACHINE = "gumstix-pxa270"
GUMSTIX_MACHINE_FEATURES ?= "wifi"
require conf/machine/gumstix-pxa270.conf
gumstix-pxa270-screen-wifi.conf:
MACHINE = "gumstix-pxa270"
GUMSTIX_MACHINE_FEATURES ?= "screen wifi"
require conf/machine/gumstix-pxa270.conf
and your local.conf.sample would read something like:
# List the features your machine requires in the list below:
# Valid features are:
# ethernet
# screen
# bluetooth
# wifi
GUMSTIX_MACHINE_FEATURES = "ethernet screen wifi bluetooth"
You might merge the gumstix-pxa270.conf into OE so OE.dev has the
maximal gumstix support but only have the raft of custom
gumstix-pxa270-*.conf files in your branch.
I'd handle flash size by having different images:
gumstix-image-16mb.bb
gumstix-image-32mb.bb
gumstix-image-64mb.bb
and a task-gumstix which defined task-gumstix-16mb etc.
If you wanted to automate the image selection with a UI, create a
symlink to the right sized flash image .bb file so "bitbake
gumstix-image" worked.
You'd also have a gumstix distro basically subclassing angstrom with
your own tweaks.
The above is all just random ideas although hopefully ones you might
fine useful. My main point is I think OE can do what you need, its just
about using its power to your advantage.
> Not a problem for us; we have nothing "proprietary". Every line of
> code we write is "open". There will always be content which isn't
> technically "proprietary" but which makes no sense outside of a
> gumstix (like say, our fbcon splash logo or something, or some of our
> default config files) which would fall in this category though probably.
Have a look at linux-rp.inc which inserts the OH logo onto the kernel if
DISTRO == "poky", even though Poky isn't in OE itself. I'm in favour of
things like that since it allows easier sharing of the files, doesn't
hurt anyone and makes people aware of the way others use the .bb file so
they take care when updating it.
> > * How do software developers use OE to write/debug code?
>
> Right now actually, this part's pretty painful. Figuring out a nice
> way to have the .bb patch-applicator interact nicely with quilt
> series files instead of having to copy/paste/sed~^~file://~;~
> $~;patch=1~ is a real PITA. I had a similar issue with buildroot
> which I basically fixed on the packages I care about by replacing
> their patch-applicator script with a call to quilt to apply my series
> files instead. Then doing actual development is just a matter of
> cd'ing to the build directory, making changes using quilt, and then
> the build scripts are automagically updated via the magic of quilt.
> I haven't yet taken a look at what'd be involved in doing that within
> OE, but if anyone's tried, or looked at it, or has any pointers on
> where to start looking, lmk.
All work directories in OE are under control of quilt and patches are
applied using it. You can cd to the build directory and play with quilt,
I do that all the time. The nasty bit is it can't use series files and
the scripts don't autoupdate (although we do have a patch resolver
shell).
It should be relatively straightforward to extend in various ways
although its obviously going to need discussion. See
classes/patch.bbclass.
> In the gumstix case, I'm thinking that the best way of dealing with
> this is to basically buffer .dev with org.gumstix.oe and
> org.gumstix.dev -- our users will "not necessarily know" that
> they're using OE. Our support wiki etc will mention that OE is under
> the hood, but it'll try and divert all support stuff to us first, not
> to openembedded.org; the danger here is that OE doesn't know what's
> going on with a big chunk of its (indirect) users, and pressure gets
> applied to effectively fork (this is what's basically happened with
> buildroot for us). On the other hand, you guys really don't want
> 7,000 gumstix users (some of whom have @aol.com email addresses and
> stuff) to start spamming the OE dev lists and bugtracker.
I think the important thing is for the OE devs to remain in close
contact with you and/or the people dealing with the 7,000 users. We
don't have the capacity to deal with them directly but we can and will
make an effort to deal with you on their behalf if you see what I mean.
Some things may require a "fork" but only hopefully in isolated areas of
code rather than the whole project. I've yet to find a need to really do
that with poky...
> > * How does OE scale to large numbers of users?
>
> Well, I think there are going to be 2 questions here actually: How
> does OE scale, and how well does monotone scale. I'm not sure
> monotone's ever had a project as large as OE with as many users pull/
> sync'ing as it might face with the OE+gumstix combination of size +
> developer base. Evidence so far shows that monotone might have some
> serious issues. The concurrency-of-access one is the most
> noticeable, given how slow monotone repo ops are. If one person is
> pull'ing and it takes 5 minutes to do so, and locks the DB so nobody
> else can pull at the same time, that's really really not going to
> work. From my subversion logs for the gumstix buildroot, we get as
> many as 400-600 "svn update"/"svn checkout" operations per day, most
> of which are during office hours in the US. The most-hit hours
> actually have over 100 in a single hour. The most-hit minutes
> frequently get dozens and sometimes (rarely) even hundreds. If those
> ops become mutually locked-out and take 5-10 minutes each now, it's
> going to seriously piss off quite a few of our customers/users. Some
> of that traffic appears to be messed-up scripts trying to pull the
> whole repo lots of times in a single minute, but we actually deal
> with that not too badly today. Getting DoS'd by those badly written
> scripts would be not good.
I worry about monotone scaling too :-(.
Cheers,
Richard
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 21:11 ` Lorn Potter
@ 2007-09-24 22:35 ` Richard Purdie
0 siblings, 0 replies; 26+ messages in thread
From: Richard Purdie @ 2007-09-24 22:35 UTC (permalink / raw)
To: openembedded-devel
On Tue, 2007-09-25 at 07:11 +1000, Lorn Potter wrote:
> > Playing devil's advocate, take Poky, for example. A person can
> > download/check it out, edit one simple configuration file
> > (comment/uncomment lines), source a file and go :).
>
> This is ok for an individual with a known device, but for a company that
> needs to produce their own distribution, for a new device, Poky is not a
> choice.
I didn't make comment about Poky's suitability for anything.
All I said is that its possible to produce a one stop configuration
process for OE which is as easy as your buildroot example. This proves
that its not some fundamental problem with OE, someone just needs to
implement it.
As for "for a company that needs to produce their own distribution, for
a new device, Poky is not a choice", thats just nonsense and our
experiences suggests otherwise ;-).
> _These_ are the people/companies that the OE/Foundation should also
> be targeting to grow.
I think all agreed that OE wants to see more of these people/companies
using OE.
Cheers,
Richard
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-24 14:48 ` Darcy Watkins
@ 2007-09-25 12:19 ` Stelios Koroneos
2007-09-25 15:15 ` Koen Kooi
2007-09-25 20:54 ` Darcy Watkins
0 siblings, 2 replies; 26+ messages in thread
From: Stelios Koroneos @ 2007-09-25 12:19 UTC (permalink / raw)
To: openembedded-devel
>
> I tried using OE, but could not get it to complete a build for PowerPC
> 405 (neither uclibc or glibc - both failed but in different ways.
> Uclibc gets cross compiler badness errors - I think it is part of this
> powerpc vs ppc stuff in Linux). After days of frustration, I had to
> bail out and use buildroot for this particular project instead.
> Obviously OE has lots of ARM and x86 users, but the PPC support falls
> behind because it isn't as popular. I'll check back for a later
> project.
>
Well, not sure when you tried it, what distro etc but the ppc405 targets in OE
work :)
Infact i am giving a presentation today at the Power.org developer conference in
Austin on powerpc/OE
As we needed a stable tree for our devices/projects we have our distro called
OPLinux and OPlinux-uclibc with gcc 4.1.1 glibc 2.5 or uclibc 0.9.28
We push the changes back to OE on a regular bases and recently provided Koen
with a 405 board so he can do more test on Angstrom which uses newer versions
of gcc/uclibc.
If you (or any other) want a copy of what we use just let me know.
Stelios
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-25 12:19 ` Stelios Koroneos
@ 2007-09-25 15:15 ` Koen Kooi
2007-09-25 20:54 ` Darcy Watkins
1 sibling, 0 replies; 26+ messages in thread
From: Koen Kooi @ 2007-09-25 15:15 UTC (permalink / raw)
To: Using the OpenEmbedded metadata to build Distributions
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Stelios Koroneos schreef:
>> I tried using OE, but could not get it to complete a build for PowerPC
>> 405 (neither uclibc or glibc - both failed but in different ways.
>> Uclibc gets cross compiler badness errors - I think it is part of this
>> powerpc vs ppc stuff in Linux). After days of frustration, I had to
>> bail out and use buildroot for this particular project instead.
>> Obviously OE has lots of ARM and x86 users, but the PPC support falls
>> behind because it isn't as popular. I'll check back for a later
>> project.
>>
>
> Well, not sure when you tried it, what distro etc but the ppc405 targets in OE
> work :)
>
> Infact i am giving a presentation today at the Power.org developer conference in
> Austin on powerpc/OE
>
> As we needed a stable tree for our devices/projects we have our distro called
> OPLinux and OPlinux-uclibc with gcc 4.1.1 glibc 2.5 or uclibc 0.9.28
> We push the changes back to OE on a regular bases and recently provided Koen
> with a 405 board so he can do more test on Angstrom which uses newer versions
> of gcc/uclibc.
> If you (or any other) want a copy of what we use just let me know.
I'm fairly sure gcc 4.1.2 and 4.2.1 are busted, they can't compile glibc, eglibc and
uclibc. I'll try to look into it next week.
regards,
Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFG+SYrMkyGM64RGpERAtUpAJ9zu8zHb9An4DJsgC/fm6AMaGcN+wCeJj/L
hXHntnJV2ZPNcYr51FmghsA=
=sx7p
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: How should OE be used?
2007-09-25 12:19 ` Stelios Koroneos
2007-09-25 15:15 ` Koen Kooi
@ 2007-09-25 20:54 ` Darcy Watkins
1 sibling, 0 replies; 26+ messages in thread
From: Darcy Watkins @ 2007-09-25 20:54 UTC (permalink / raw)
To: openembedded-devel
Hi Stelios,
I did the following:
Updated my copy to OE upstreams using monotone
Set my distro to oplinux-uclibc
Set my machine to dht-walnut
Let the distro and machine setup ARCH and TARGET_OS as suggested.
The build didn't complete...
adjtimex.c:14 error '__adjtimex' aliased to undefined symbol
'adjtimex'
adjtimex.c:15 error 'ntp_adjtime' aliased to undefined symbol
'adjtimex'
Somewhere inside uclibc task do_compile
Are you using a different machine file?
I am using an AMCC Taihu but I'll be happy just to be pointed to any
PPC405 machine / distro that builds without errors as a start.
Darcy
>As we needed a stable tree for our devices/projects we have
>our distro called
>OPLinux and OPlinux-uclibc with gcc 4.1.1 glibc 2.5 or uclibc 0.9.28
>We push the changes back to OE on a regular bases and recently
>provided Koen
>with a 405 board so he can do more test on Angstrom which uses
>newer .versions
>of gcc/uclibc.
>If you (or any other) want a copy of what we use just let me know.
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2007-09-25 20:59 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-09-22 21:24 How should OE be used? Philip Balister
2007-09-23 11:19 ` Leon Woestenberg
2007-09-23 12:33 ` Koen Kooi
2007-09-24 17:20 ` Tim Bird
2007-09-24 17:43 ` Koen Kooi
2007-09-24 18:01 ` Tim Bird
2007-09-24 15:55 ` Tim Bird
2007-09-24 16:35 ` Philip Balister
2007-09-24 17:14 ` Tim Bird
2007-09-24 18:04 ` Leon Woestenberg
2007-09-24 19:23 ` Koen Kooi
2007-09-24 14:48 ` Darcy Watkins
2007-09-25 12:19 ` Stelios Koroneos
2007-09-25 15:15 ` Koen Kooi
2007-09-25 20:54 ` Darcy Watkins
2007-09-24 16:14 ` Cliff Brake
2007-09-24 18:36 ` Craig Hughes
2007-09-24 19:27 ` Philip Balister
2007-09-24 21:37 ` Richard Purdie
2007-09-24 18:48 ` Lorn Potter
2007-09-24 19:01 ` Craig Hughes
2007-09-24 19:39 ` Lorn Potter
2007-09-24 20:33 ` Craig Hughes
2007-09-24 20:52 ` Richard Purdie
2007-09-24 21:11 ` Lorn Potter
2007-09-24 22:35 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.