* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Sam Ravnborg @ 2008-06-12 17:23 UTC (permalink / raw)
To: Bill Gatliff; +Cc: linux-embedded
In-Reply-To: <48515978.8040603@billgatliff.com>
On Thu, Jun 12, 2008 at 12:14:32PM -0500, Bill Gatliff wrote:
> Paul Mundt wrote:
>
> > Yes, that's the easy case. It's things like perl that are the corner
> > cases, and my objection comes from the fact that people think we ought to
> > not have the kernel depend on perl rather than just fixing the package
> > itself. Autoconf/libtool damage is an entirely different problem :-)
> >
>
> At first glance, it seems like checkincludes.pl could be duplicated by egrep |
> uniq | wc vs. egrep | wc. Not quite sure what checkversion.pl is trying to do.
>
> The namespace.pl script looks optional, as does export_report.pl.
>
> So maybe we could _reduce_ dependency on perl, if there's any advantage to gain
> by doing so. But the kernel build machinery isn't dependent on very many other
> systems (just tcl, bash and gcc-core), so I don't really see the point unless
> you could completely eliminate perl. And I don't see how you might do *that*
> without dragging in a bunch of stuff to replace it, thereby increasing the
> number of dependencies.
All the noise about the perl dependency of the kernel build are
rooted in two things:
1) That we now have a mandatory part of the build that uses perl (see kernel/Makefile)
2) That I told that I consider rewriting the core of the build system in perl
As for 1) I have seen a patch submitted once. And I do not hear many complaint either,
albeit Rob Landley is a bit loud here (and he was also the one submitting the patch).
That patch was not acceptable as is - and noone has updated it.
As for 2) then let see if that ever happens ;-)
Sam
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Mike Frysinger @ 2008-06-12 17:22 UTC (permalink / raw)
To: Bill Gatliff; +Cc: linux-embedded
In-Reply-To: <48515978.8040603@billgatliff.com>
On Thu, Jun 12, 2008 at 1:14 PM, Bill Gatliff wrote:
> Paul Mundt wrote:
>> Yes, that's the easy case. It's things like perl that are the corner
>> cases, and my objection comes from the fact that people think we ought to
>> not have the kernel depend on perl rather than just fixing the package
>> itself. Autoconf/libtool damage is an entirely different problem :-)
>
> At first glance, it seems like checkincludes.pl could be duplicated by egrep |
> uniq | wc vs. egrep | wc. Not quite sure what checkversion.pl is trying to do.
>
> The namespace.pl script looks optional, as does export_report.pl.
>
> So maybe we could _reduce_ dependency on perl, if there's any advantage to gain
> by doing so. But the kernel build machinery isn't dependent on very many other
> systems (just tcl, bash and gcc-core), so I don't really see the point unless
> you could completely eliminate perl. And I don't see how you might do *that*
> without dragging in a bunch of stuff to replace it, thereby increasing the
> number of dependencies.
the idea is to keep perl from being required to build the kernel
itself. tcl is not required in this case either. if you want to use
crazy languages to optional things above and beyond, then no one is
going to complain (and if they do, well screw them).
-mike
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bill Gatliff @ 2008-06-12 17:19 UTC (permalink / raw)
To: linux-embedded
In-Reply-To: <20080612170108.GA17231@cs181133002.pp.htv.fi>
Adrian Bunk wrote:
> There's also the middle way taken by e.g. Scratchbox of using
> cross-compilers and other tools from the machine the compilation
> is done on, but emulating a native build for the software being
> compiled.
I suppose. But I've never gotten Scratchbox to work for me, at all.
Whether the problem is me or Scratchbox, I don't really want to know. :)
b.g.
--
Bill Gatliff
bgat@billgatliff.com
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bill Gatliff @ 2008-06-12 17:17 UTC (permalink / raw)
To: linux-embedded
In-Reply-To: <1213288655.26255.168.camel@pmac.infradead.org>
David Woodhouse wrote:
> Yeah, I was building Red Hat Linux packages for sh3 many years ago,
> using tricks like that. But there was always _something_ else going
> wrong, however much you hacked around it. And a lot of it would only
> turn up at runtime, not build time. I would never consider shipping a
> product with a large number of userspace packages cross-compiled.
I take it on a case-by-case basis. But yea, validation is definitely a pain.
Especially with all the silent failures (i.e. getting endianness wrong) that the
configure script creates for you.
> For minimal file systems with a select handful of tools which can be
> tested exhaustively, it's not so bad. But for any 'full-featured'
> userspace, I think cross-compilation is completely insane.
We both agree that it'll *drive* you insane, at least. And I'm 90% with you on
the rest. :)
b.g.
--
Bill Gatliff
bgat@billgatliff.com
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bill Gatliff @ 2008-06-12 17:14 UTC (permalink / raw)
To: linux-embedded
In-Reply-To: <20080612163155.GC9327@linux-sh.org>
Paul Mundt wrote:
> Yes, that's the easy case. It's things like perl that are the corner
> cases, and my objection comes from the fact that people think we ought to
> not have the kernel depend on perl rather than just fixing the package
> itself. Autoconf/libtool damage is an entirely different problem :-)
>
At first glance, it seems like checkincludes.pl could be duplicated by egrep |
uniq | wc vs. egrep | wc. Not quite sure what checkversion.pl is trying to do.
The namespace.pl script looks optional, as does export_report.pl.
So maybe we could _reduce_ dependency on perl, if there's any advantage to gain
by doing so. But the kernel build machinery isn't dependent on very many other
systems (just tcl, bash and gcc-core), so I don't really see the point unless
you could completely eliminate perl. And I don't see how you might do *that*
without dragging in a bunch of stuff to replace it, thereby increasing the
number of dependencies.
Any machine beefy enough to build the kernel is more than adequate to host perl
as well. So maybe the "corner case" is situations where you're trying to build
the kernel on an overly-minimalist host--- perhaps one that you constructed
yourself using linux-from-scratch?
Not trying to fan the flames any, of course. I'm just sayin'...
The worst for cross-building so far for me has been openssh. And even that's
tolerable once you find your way around the autoconf stuff. But this would be
the wrong list for that. :)
b.g.
--
Bill Gatliff
bgat@billgatliff.com
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Adrian Bunk @ 2008-06-12 17:01 UTC (permalink / raw)
To: David Woodhouse
Cc: Bill Gatliff, Paul Mundt, Tim Bird, Rob Landley, Greg Ungerer,
Sam Ravnborg, Leon Woestenberg, linux-embedded
In-Reply-To: <1213288655.26255.168.camel@pmac.infradead.org>
On Thu, Jun 12, 2008 at 05:37:35PM +0100, David Woodhouse wrote:
> On Thu, 2008-06-12 at 11:28 -0500, Bill Gatliff wrote:
> > Guys:
> >
> > > If you opt to cross-compile, having to deal with those
> > > sorts of things is the price you pay.
> >
> >
> > If the build system derives from autoconf, then a hacked-up config.cache (or
> > equivalent command-line args) often solves problems for me. Just give the cache
> > the answers that it would otherwise have to get by running code on the target
> > machine.
> >
> > That's how emdebian is doing a bunch of their stuff, and I have to admit that it
> > works pretty darned well. It's also handy for configuration management, since
> > the cache file itself is plaintext and therefore svn/git/bzr/cvs/...-friendly.
>
> Yeah, I was building Red Hat Linux packages for sh3 many years ago,
> using tricks like that. But there was always _something_ else going
> wrong, however much you hacked around it. And a lot of it would only
> turn up at runtime, not build time. I would never consider shipping a
> product with a large number of userspace packages cross-compiled.
>
> For minimal file systems with a select handful of tools which can be
> tested exhaustively, it's not so bad. But for any 'full-featured'
> userspace, I think cross-compilation is completely insane.
There's also the middle way taken by e.g. Scratchbox of using
cross-compilers and other tools from the machine the compilation
is done on, but emulating a native build for the software being
compiled.
This works quite good in practice.
> dwmw2
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Mike Frysinger @ 2008-06-12 16:38 UTC (permalink / raw)
To: Paul Mundt
Cc: Bill Gatliff, David Woodhouse, Tim Bird, Rob Landley,
Greg Ungerer, Sam Ravnborg, Leon Woestenberg, linux-embedded
In-Reply-To: <20080612163155.GC9327@linux-sh.org>
On Thu, Jun 12, 2008 at 12:31 PM, Paul Mundt wrote:
> On Thu, Jun 12, 2008 at 11:28:10AM -0500, Bill Gatliff wrote:
>> > If you opt to cross-compile, having to deal with those
>> > sorts of things is the price you pay.
>>
>>
>> If the build system derives from autoconf, then a hacked-up config.cache (or
>> equivalent command-line args) often solves problems for me. Just give the cache
>> the answers that it would otherwise have to get by running code on the target
>> machine.
>>
>> That's how emdebian is doing a bunch of their stuff, and I have to admit that it
>> works pretty darned well. It's also handy for configuration management, since
>> the cache file itself is plaintext and therefore svn/git/bzr/cvs/...-friendly.
>
> Yes, that's the easy case. It's things like perl that are the corner
> cases, and my objection comes from the fact that people think we ought to
> not have the kernel depend on perl rather than just fixing the package
> itself. Autoconf/libtool damage is an entirely different problem :-)
of the core packages, perl and openssl tend to be heavily damaged.
openssl because it depends on perl instead of a real build system.
-mike
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: David Woodhouse @ 2008-06-12 16:37 UTC (permalink / raw)
To: Bill Gatliff
Cc: Paul Mundt, Tim Bird, Rob Landley, Greg Ungerer, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <48514E9A.3080901@billgatliff.com>
On Thu, 2008-06-12 at 11:28 -0500, Bill Gatliff wrote:
> Guys:
>
> > If you opt to cross-compile, having to deal with those
> > sorts of things is the price you pay.
>
>
> If the build system derives from autoconf, then a hacked-up config.cache (or
> equivalent command-line args) often solves problems for me. Just give the cache
> the answers that it would otherwise have to get by running code on the target
> machine.
>
> That's how emdebian is doing a bunch of their stuff, and I have to admit that it
> works pretty darned well. It's also handy for configuration management, since
> the cache file itself is plaintext and therefore svn/git/bzr/cvs/...-friendly.
Yeah, I was building Red Hat Linux packages for sh3 many years ago,
using tricks like that. But there was always _something_ else going
wrong, however much you hacked around it. And a lot of it would only
turn up at runtime, not build time. I would never consider shipping a
product with a large number of userspace packages cross-compiled.
For minimal file systems with a select handful of tools which can be
tested exhaustively, it's not so bad. But for any 'full-featured'
userspace, I think cross-compilation is completely insane.
--
dwmw2
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Paul Mundt @ 2008-06-12 16:31 UTC (permalink / raw)
To: Bill Gatliff
Cc: David Woodhouse, Tim Bird, Rob Landley, Greg Ungerer,
Sam Ravnborg, Leon Woestenberg, linux-embedded
In-Reply-To: <48514E9A.3080901@billgatliff.com>
On Thu, Jun 12, 2008 at 11:28:10AM -0500, Bill Gatliff wrote:
> Guys:
>
> > If you opt to cross-compile, having to deal with those
> > sorts of things is the price you pay.
>
>
> If the build system derives from autoconf, then a hacked-up config.cache (or
> equivalent command-line args) often solves problems for me. Just give the cache
> the answers that it would otherwise have to get by running code on the target
> machine.
>
> That's how emdebian is doing a bunch of their stuff, and I have to admit that it
> works pretty darned well. It's also handy for configuration management, since
> the cache file itself is plaintext and therefore svn/git/bzr/cvs/...-friendly.
>
Yes, that's the easy case. It's things like perl that are the corner
cases, and my objection comes from the fact that people think we ought to
not have the kernel depend on perl rather than just fixing the package
itself. Autoconf/libtool damage is an entirely different problem :-)
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bill Gatliff @ 2008-06-12 16:28 UTC (permalink / raw)
To: Paul Mundt
Cc: David Woodhouse, Tim Bird, Rob Landley, Greg Ungerer,
Sam Ravnborg, Leon Woestenberg, linux-embedded
In-Reply-To: <20080612160845.GB9327@linux-sh.org>
Guys:
> If you opt to cross-compile, having to deal with those
> sorts of things is the price you pay.
If the build system derives from autoconf, then a hacked-up config.cache (or
equivalent command-line args) often solves problems for me. Just give the cache
the answers that it would otherwise have to get by running code on the target
machine.
That's how emdebian is doing a bunch of their stuff, and I have to admit that it
works pretty darned well. It's also handy for configuration management, since
the cache file itself is plaintext and therefore svn/git/bzr/cvs/...-friendly.
b.g.
--
Bill Gatliff
bgat@billgatliff.com
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Tim Bird @ 2008-06-12 16:23 UTC (permalink / raw)
To: David Woodhouse
Cc: Rob Landley, Greg Ungerer, Sam Ravnborg, Leon Woestenberg,
linux-embedded
In-Reply-To: <1213285831.26255.152.camel@pmac.infradead.org>
David Woodhouse wrote:
>>... fixing
>> them in the upstream packages (or in the autoconf system itself).
>>
>> Once someone fixes the cross-compilation issues for a package, they usually
>> stay fixed, if the fixes are mainlined.
>
> I don't think that's true, unfortunately. Autoconf makes it _easy_ to do
> the wrong thing, and people will often introduce new problems.
If autoconf is the problem (and I think it is), then that's what
should be fixed (see my original post). At a minimum, it would be
nice if it had more built-in detection and warning of techniques
that were dangerous for cross-compilation.
While I was writing the above,
Paul Mundt wrote:
> You can
> either try to fix the packages in question, convince the package
> developers to rip out the parts that cause trouble for your environment,
> fix your own build environment to meet the needs of the packages, or
> whine about it on a mailing list. Empirically we already know which one
> of those options is going to win out. ;-)
LOL. Well, at least Rob has put his money where his mouth is (so to
speak) with Firmware Linux. The chance that I'll do anything but whine
about autoconf is slim indeed... I'll shut up now!
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Mike Frysinger @ 2008-06-12 16:15 UTC (permalink / raw)
To: David Woodhouse
Cc: Tim Bird, Rob Landley, Greg Ungerer, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <1213286929.26255.161.camel@pmac.infradead.org>
On Thu, Jun 12, 2008 at 12:08 PM, David Woodhouse wrote:
> On Thu, 2008-06-12 at 12:05 -0400, Mike Frysinger wrote:
>> people cant even write proper *native* makefiles. mtd-utils for
>> example ;).
>
> Criticism in 'diff -u' form preferred :)
i sent via `git-send-email` yesterday to the mtd lists ... either it
got silently dropped, or someone hates me
-mike
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Robert P. J. Day @ 2008-06-12 16:12 UTC (permalink / raw)
To: Mike Frysinger
Cc: David Woodhouse, Tim Bird, Rob Landley, Greg Ungerer,
Sam Ravnborg, Leon Woestenberg, linux-embedded
In-Reply-To: <8bd0f97a0806120905u532ce5f9sf231e185173de791@mail.gmail.com>
On Thu, 12 Jun 2008, Mike Frysinger wrote:
> On Thu, Jun 12, 2008 at 11:50 AM, David Woodhouse wrote:
> > If we just made people write portable code and proper Makefiles,
> > it would be less of an issue :)
>
> people cant even write proper *native* makefiles. mtd-utils for
> example ;).
meooowww! :-) but at the risk of dragging this even further
off-topic, i am *constantly* asked by people how to set up makefiles
for their software project, and what would be nice is a small
collection of examples of a makefile (or makefiles) done *right*. as
in, properly recursive, supports cross-compiling without having to
remove your left nad, etc, etc.
so ... would anyone like to recommend a software package or two
somewhere whose makefile(s) is/are, ITHO, done well? that i/we could
just point at and say, "do it like that!" anyone? anyone? bueller?
rday
--
========================================================================
Robert P. J. Day
Linux Consulting, Training and Annoying Kernel Pedantry:
Have classroom, will lecture.
http://crashcourse.ca Waterloo, Ontario, CANADA
========================================================================
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: David Woodhouse @ 2008-06-12 16:08 UTC (permalink / raw)
To: Mike Frysinger
Cc: Tim Bird, Rob Landley, Greg Ungerer, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <8bd0f97a0806120905u532ce5f9sf231e185173de791@mail.gmail.com>
On Thu, 2008-06-12 at 12:05 -0400, Mike Frysinger wrote:
> people cant even write proper *native* makefiles. mtd-utils for
> example ;).
Criticism in 'diff -u' form preferred :)
--
dwmw2
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Paul Mundt @ 2008-06-12 16:08 UTC (permalink / raw)
To: David Woodhouse
Cc: Tim Bird, Rob Landley, Greg Ungerer, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <1213285831.26255.152.camel@pmac.infradead.org>
On Thu, Jun 12, 2008 at 04:50:31PM +0100, David Woodhouse wrote:
> On Thu, 2008-06-12 at 08:23 -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > However, having one or more full-time engineers devoted to debugging
> > > cross-compile issues is quite a high price to pay too. Moore's law really
> > > doesn't help that one.
> > >
> > > I'm not saying either solution is perfect, I'm just saying the "build under
> > > emulation" approach is a viable alternative that gets more attractive as time
> > > passes, both because of ongoing development on emulators and because of
> > > Moore's law on the hardware.
> >
> > I agree with much that you have said, Rob, and I understand the argument
> > for getting the most gain from the least resources, but I have a philosophical
> > problem with working around the cross-compilation problems instead of fixing
> > them in the upstream packages (or in the autoconf system itself).
> >
> > Once someone fixes the cross-compilation issues for a package, they usually
> > stay fixed, if the fixes are mainlined.
>
> I don't think that's true, unfortunately. Autoconf makes it _easy_ to do
> the wrong thing, and people will often introduce new problems.
>
> If we just made people write portable code and proper Makefiles, it
> would be less of an issue :)
>
The other issue is that people that are working in this space tend to do
very little beyond solving their immediate troubles. Since perl was
mentioned, it also makes a good example. Embedded distros have been
cross-compiling perl for pretty much the last decade, yet even today
people are having the exact same issues and acting as this is some sort
of surprise that needs to be worked arond, rather than treating it as a
solved problem. If you opt to cross-compile, having to deal with those
sorts of things is the price you pay.
Holding the kernel hostage to this kind of brain-damage is simply beyond
ridiculous. There are a lot of things outside of the kernel that have a
dependency on perl too, how much time do people want to spend trying to
fix the build system to match their environment before they realize their
environment needs to scale to match the build environment that everyone
else is already using?
Building under qemu in a "native" environment is one way to work around
this problem, and a lot of companies have opted for that approach rather
than trying to fix the problematic packages. If you aren't building
natively, your options are effectively limited by convention. You can
either try to fix the packages in question, convince the package
developers to rip out the parts that cause trouble for your environment,
fix your own build environment to meet the needs of the packages, or
whine about it on a mailing list. Empirically we already know which one
of those options is going to win out. ;-)
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Tim Bird @ 2008-06-12 16:07 UTC (permalink / raw)
To: Sam Ravnborg
Cc: Geert Uytterhoeven, linux-embedded, linux kernel, David Woodhouse,
Holger Schurig
In-Reply-To: <20080612131931.GB13702@uranus.ravnborg.org>
Sam Ravnborg wrote:
> Hi Tim.
>
>> The program is also now better structured, IMHO.
>> -- Tim
>
> Seeing this programs gets frequent updates (good!) I have not
> yet applied it.
>
> When you consider it stabilized could you please drop me a
> new mail including full changelog and updated patch.
>
> And please cc: linux-kbuild@vger.kernel.org + linux-kernel on the
> submission.
Will do. I'll give it a little settle time, and then
notify you.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Mike Frysinger @ 2008-06-12 16:05 UTC (permalink / raw)
To: David Woodhouse
Cc: Tim Bird, Rob Landley, Greg Ungerer, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <1213285831.26255.152.camel@pmac.infradead.org>
On Thu, Jun 12, 2008 at 11:50 AM, David Woodhouse wrote:
> On Thu, 2008-06-12 at 08:23 -0700, Tim Bird wrote:
>> Rob Landley wrote:
>> > However, having one or more full-time engineers devoted to debugging
>> > cross-compile issues is quite a high price to pay too. Moore's law really
>> > doesn't help that one.
>> >
>> > I'm not saying either solution is perfect, I'm just saying the "build under
>> > emulation" approach is a viable alternative that gets more attractive as time
>> > passes, both because of ongoing development on emulators and because of
>> > Moore's law on the hardware.
>>
>> I agree with much that you have said, Rob, and I understand the argument
>> for getting the most gain from the least resources, but I have a philosophical
>> problem with working around the cross-compilation problems instead of fixing
>> them in the upstream packages (or in the autoconf system itself).
>>
>> Once someone fixes the cross-compilation issues for a package, they usually
>> stay fixed, if the fixes are mainlined.
>
> I don't think that's true, unfortunately. Autoconf makes it _easy_ to do
> the wrong thing, and people will often introduce new problems.
>
> If we just made people write portable code and proper Makefiles, it
> would be less of an issue :)
people cant even write proper *native* makefiles. mtd-utils for example ;).
-mike
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: David Woodhouse @ 2008-06-12 15:50 UTC (permalink / raw)
To: Tim Bird
Cc: Rob Landley, Greg Ungerer, Sam Ravnborg, Leon Woestenberg,
linux-embedded
In-Reply-To: <48513F5A.6010008@am.sony.com>
On Thu, 2008-06-12 at 08:23 -0700, Tim Bird wrote:
> Rob Landley wrote:
> > However, having one or more full-time engineers devoted to debugging
> > cross-compile issues is quite a high price to pay too. Moore's law really
> > doesn't help that one.
> >
> > I'm not saying either solution is perfect, I'm just saying the "build under
> > emulation" approach is a viable alternative that gets more attractive as time
> > passes, both because of ongoing development on emulators and because of
> > Moore's law on the hardware.
>
> I agree with much that you have said, Rob, and I understand the argument
> for getting the most gain from the least resources, but I have a philosophical
> problem with working around the cross-compilation problems instead of fixing
> them in the upstream packages (or in the autoconf system itself).
>
> Once someone fixes the cross-compilation issues for a package, they usually
> stay fixed, if the fixes are mainlined.
I don't think that's true, unfortunately. Autoconf makes it _easy_ to do
the wrong thing, and people will often introduce new problems.
If we just made people write portable code and proper Makefiles, it
would be less of an issue :)
--
dwmw2
^ permalink raw reply
* cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Tim Bird @ 2008-06-12 15:23 UTC (permalink / raw)
To: Rob Landley
Cc: Greg Ungerer, Sam Ravnborg, Leon Woestenberg, David Woodhouse,
linux-embedded
In-Reply-To: <200806111941.51221.rob@landley.net>
Rob Landley wrote:
> However, having one or more full-time engineers devoted to debugging
> cross-compile issues is quite a high price to pay too. Moore's law really
> doesn't help that one.
>
> I'm not saying either solution is perfect, I'm just saying the "build under
> emulation" approach is a viable alternative that gets more attractive as time
> passes, both because of ongoing development on emulators and because of
> Moore's law on the hardware.
I agree with much that you have said, Rob, and I understand the argument
for getting the most gain from the least resources, but I have a philosophical
problem with working around the cross-compilation problems instead of fixing
them in the upstream packages (or in the autoconf system itself).
Once someone fixes the cross-compilation issues for a package, they usually
stay fixed, if the fixes are mainlined.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Sam Ravnborg @ 2008-06-12 13:19 UTC (permalink / raw)
To: Tim Bird
Cc: Geert Uytterhoeven, linux-embedded, linux kernel, David Woodhouse,
Holger Schurig
In-Reply-To: <48501836.10900@am.sony.com>
Hi Tim.
> The program is also now better structured, IMHO.
> -- Tim
Seeing this programs gets frequent updates (good!) I have not
yet applied it.
When you consider it stabilized could you please drop me a
new mail including full changelog and updated patch.
And please cc: linux-kbuild@vger.kernel.org + linux-kernel on the
submission.
Thanks,
Sam
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Jamie Lokier @ 2008-06-12 7:55 UTC (permalink / raw)
To: Rob Landley
Cc: Greg Ungerer, Sam Ravnborg, Leon Woestenberg, David Woodhouse,
linux-embedded
In-Reply-To: <200806111941.51221.rob@landley.net>
Rob Landley wrote:
> Most packages don't cross compile at all. Debian has somewhere north of
> 30,000 packages. Every project that does large scale cross compiling
> (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends
> to have about 200 packages that cross compile more or less easily, another
> 400 or so that can be made to cross compile with _lot_ of effort and a large
> enough rock, and then the project stalls at about that size.
+1.
I spent several months fixing up cross-compile issues on Gentoo
Embedded a few years ago, for a specific application - only a small
subset of packages needed. The majority of packages I needed failed
to compile out of the box, one way or another - Glibc dependencies,
arch dependencies, scripts which depend on the host environment, or
invoke the host compiler, or the host Perl (and then depend on it's
byte order). etc.
More recently, I've been compiling a build which is _intended_ for
cross-compilation - it's an old uClinux kit, patched by a third party.
Even that fails to build on newer GNU/Linuxes, as the syntax of GNU
Make has changed, and Bash has changed. Also GCC 2.old doesn't
compile on current GNU/Linux with GCC 4.new.
Fortunately the latter were a few small, easy to fix issues. But I
understand now why some find it important to have a replicatable build
environment when you need to get an old distribution out of the closet
to update firmware for some 5 year old device.
Virtual machines ought to be great for that. They are. But even
those are surprisingly changable - images that worked on a VM a few
years ago no longer work on the current version of the VM host.
-- Jamie
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-12 1:42 UTC (permalink / raw)
To: Tim Bird
Cc: Adrian Bunk, Sam Ravnborg, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <48502BAF.7060209@am.sony.com>
On Wednesday 11 June 2008 14:46:55 Tim Bird wrote:
> Adrian Bunk wrote:
> > Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> > April 2006.
>
> Well, dangit there it is!
>
> The patch I googled had it going into Documentation/kbuild. It
> somehow escaped my attention in the README.
Mine as well. (I grepped for KCONFIG_ALLCONFIG in Documentation, not in the
rest of the tree. I of all people should know better...)
> If I was
> a little more skilled with my grep-ing I would have found it.
> Sorry for the bother!
The linux kernel has documentation in Documentation, in the "make htmldocs"
output, in the Kconfig help entries, in README files, in the output of "make
help", and several other places. (And then there's all the stuff that's not
_in_ the kernel tarball...)
It's easy to miss.
Rob
P.S: yes, README files plural.
find . -name "*[Rr][Ee][Aa][Dd][Mm][Ee]*" | grep -v Documentation | wc
69 69 2315
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Rob Landley @ 2008-06-12 0:41 UTC (permalink / raw)
To: Greg Ungerer
Cc: Sam Ravnborg, Leon Woestenberg, David Woodhouse, linux-embedded
In-Reply-To: <484F66F8.4020409@snapgear.com>
On Wednesday 11 June 2008 00:47:36 Greg Ungerer wrote:
> Hi Rob,
>
> Rob Landley wrote:
> > On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote:
> >>> (Maybe I _am_ the only person who still cares about
> >>> building on a host without perl. If I wasn't, somebody else would have
> >>> acked the patch...)
> >>
> >> perl is pretty standard
> >
> > An implementation is not the same thing as a standard. If you mean
> > "there is one implementation everybody uses, ala excel and Word, and even
> > the Perl guys can't reproduce it from scratch as parrot showed", then
> > you're using a different definition of the word "standard" than I am.
> >
> > Or do you mean it comes preinstalled on most modern systems, the way
> > Windows does, and who could object to that?
> >
> > I know from experience that it's an _amazing_ pain to try to cross
> > compile the sucker...
>
> Ain't that the truth!
>
> >> and I fail to see the benefits of avoiding it.
> >> For embedded development I see even less benefits as I assume
> >> any sane embedded development environment are based on a
> >> cross-toolchain so you do the build on a high perfomance box.
> >>
> >> Building everything for my arm board on the arm board would be a disater
> >> for example.
> >
> > I build everything for my arm board natively, on an arm system running
> > under qemu, calling out to the cross compiler via distcc to accelerate
> > the limited parts of the process that cross compiling doesn't actually
> > break. To get to that point, I cross compile just enough to get a native
> > development environment, and then I avoid cross compiling from then on
> > because it's an enormous source of complexity and random breakage.
>
> Random breakage?
Cross compiling breaks stuff, yes.
Most packages don't cross compile at all. Debian has somewhere north of
30,000 packages. Every project that does large scale cross compiling
(buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends
to have about 200 packages that cross compile more or less easily, another
400 or so that can be made to cross compile with _lot_ of effort and a large
enough rock, and then the project stalls at about that size.
> > I did this because throwing hardware at the problem is cheaper than
> > throwing engineering time at the problem, because Moore's Law is on my
> > side, and
>
> Are you sure about that?
> How well does qemu do SMP? At all?
QEMU does not currently do SMP at all. There are proposals to make qemu
multi-threaded and handle SMP that way, but they're at least a year away from
anybody actually trying to implement them. (The big destabilization right
now is the new code generator, and that's plenty for the moment. Anybody who
wanted to wave money at codesourcery could probably get it implemented on a
deadline for their platforms of interest, but in the absence of that it's not
really a priority right now.)
Distcc can take advantage of smp, but that won't help the ./configure stage
and I need to do some work on distcc to teach it to understand more gcc
command lines. (For one thing, distcc can't break "compile and link"
commands ala "gcc hello.c" into separate "compile" and "link" stages, this it
can't distribute those. This takes out pretty much the entire uClibc build,
for example. But the gcc build parallelizes quite well.)
> Modern x86 and friends are getting most new performance from
> more cores. A cross compile today can take advantage of those
> for the most part. Your emulated system probably can't.
It can if you're using distcc to call out to the cross compiler, which my
scripts do (./emulator-build.sh does it with the build/* stuff
and ./run-with-distcc.sh does it for the shipped system-image tarballs).
Some of the other build systems out there hook qemu application emulation up
to the kernel's misc binary support so a ./configure that builds arm
executables can run them. (Openembedded, was it? Open moko? Something like
that.) But this only solves one of about a dozen major problems cross
compiling is prone to. Building natively solves all of 'em, except for
speed.
> An order of magnitude compile time (or more) for a native build
> is quite a high price to pay :-)
Yup. Agreed. No argument there.
However, having one or more full-time engineers devoted to debugging
cross-compile issues is quite a high price to pay too. Moore's law really
doesn't help that one.
I'm not saying either solution is perfect, I'm just saying the "build under
emulation" approach is a viable alternative that gets more attractive as time
passes, both because of ongoing development on emulators and because of
Moore's law on the hardware.
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-12 0:01 UTC (permalink / raw)
To: Tim Bird
Cc: Sam Ravnborg, Adrian Bunk, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <48502304.1030304@am.sony.com>
On Wednesday 11 June 2008 14:09:56 Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >>
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> >
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
>
> Indeed. I had never heard of it, but it appears to have been
> around for at least 18 months. It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
Nor did the documentation I posted back in November 2005:
http://lwn.net/Articles/160497/
Nor its resubmission:
http://lwn.net/Articles/161086/
Nor its re-resubmission:
http://lkml.org/lkml/2006/7/6/404
Randy's submission was after those three...
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Sam Ravnborg @ 2008-06-11 19:48 UTC (permalink / raw)
To: Adrian Bunk
Cc: Tim Bird, Rob Landley, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080611193639.GB2958@cs181133002.pp.htv.fi>
On Wed, Jun 11, 2008 at 10:36:39PM +0300, Adrian Bunk wrote:
> On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> > Sam Ravnborg wrote:
> > >>> When I get my kconfig patchset polished you will be able to do:
> > >>>
> > >>> make K=my_mini_config allnoconfig
> > >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > >
> > > Somehow yes. The K= notation I hope will see more use. Only very few
> > > people know the KCONFIG_ALL*CONFIG= trick.
> >
> > Indeed. I had never heard of it, but it appears to have been
> > around for at least 18 months. It looks like Randy Dunlap tried
> > to submit some documentation for this in Oct 2006, but it didn't
> > make it in?
> >...
>
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.
The one I refer to is not.
Sam
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox