* Re: Recommendation for activating a deferred module init in the kernel
From: Jörn Engel @ 2008-06-17 19:07 UTC (permalink / raw)
To: Tim Bird; +Cc: linux-embedded
In-Reply-To: <48580116.9070504@am.sony.com>
On Tue, 17 June 2008 11:23:18 -0700, Tim Bird wrote:
>
> I'm not that happy using an ioctl for this trigger. What is
> the preferred method of activating a kernel feature like this?
> I presume something in /proc or /sys, but I'm not sure.
I personally would be unhappy with any kind of interface for this. It
would be much nicer to make it transparent and still get the benefits.
One option would be to start a kernel thread for the initialization and
renice it to 19 or so.
If you want an explicit trigger, you could either hook into init_post()
or have hooks in the open functions of drivers with deferred
initialization. Obviously you need to wait for completion here anyway,
so adding a trigger wouldn't be too expensive.
Jörn
--
Joern's library part 13:
http://www.chip-architect.com/
--
To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Recommendation for activating a deferred module init in the kernel
From: David VomLehn @ 2008-06-17 18:51 UTC (permalink / raw)
To: Tim Bird; +Cc: linux-embedded
In-Reply-To: <48580116.9070504@am.sony.com>
Tim Bird wrote:
> Hi all,
>
> I am working with a product team on bootup time issues. One technique that we
> are forward-porting from an old kernel (and that I thought I might work on
> mainlining) is to compile modules statically into the kernel, but defer their
> initialization until after boot time.
...
> One of the main sub-systems that we defer initialization of this way is USB,
> and this saves quite a bit of time. (Of course the same, or slightly more CPU
> cycles are eventually used during bootup time. But this lets us get to user
> space quicker so we can start user-visible applications faster.)
>
> I'm not that happy using an ioctl for this trigger. What is the preferred
> method of activating a kernel feature like this? I presume something in /proc
> or /sys, but I'm not sure.
From your description, it seems as though you always want to do the
initialization, you just may want to delay it until well after starting up user
space. If something like this were performance-critical, an ioctl might be
justified, but this would be a one-time trigger. I think that making this as
closely analogous to loading a kernel module as possible is a good conceptual
approach. Since you might choose, at run time, whether or not to load a kernel
module, it makes sense to do this on a device-by-device basis. To me, that
suggests doing something in /sys.
My first impression was that this was an awful kludge, but drawing the parallel
to loadable modules makes me not only happier, but also leads me into wondering
if there shouldn't be a generic framework for supporting this. So, instead of
using module_init, there might be some other macro that indicated that this
driver was to be initialized in a deferred, and optional, fashion.
--
David VomLehn, dvomlehn@cisco.com
The opinions expressed herein are likely mine, but might not be my employer's...
- - - - - Cisco - - - - -
This e-mail and any attachments may contain information which is confidential,
proprietary, privileged or otherwise protected by law. The information is solely
intended for the named addressee (or a person responsible for delivering it to
the addressee). If you are not the intended recipient of this message, you are
not authorized to read, print, retain, copy or disseminate this message or any
part of it. If you have received this e-mail in error, please notify the sender
immediately by return e-mail and delete it from your computer.
^ permalink raw reply
* Recommendation for activating a deferred module init in the kernel
From: Tim Bird @ 2008-06-17 18:23 UTC (permalink / raw)
To: linux-embedded
Hi all,
I am working with a product team on bootup time issues. One technique
that we are forward-porting from an old kernel (and that I thought I
might work on mainlining) is to compile modules statically into the kernel,
but defer their initialization until after boot time.
Normally, module init routines are declared with module_init(), which,
when they are statically linked into the kernel, uses the macro
__define_initcall() to add an entry to a special init segment.
This is called during bootup by do_initcall() (in init/main.c),
and the memory where the function resides is eventually freed.
There are several phases of initcalls (see include/linux/init.h),
including core, postcore, arch, subsys, fs, rootfs, device, and late.
In our modification, we:
1) add another phase for deferred initcalls
2) modify the module_init macro definition to use that phase, for the
modules we wish to defer initialization of
3) modify the free code to not free the memory until later
4) add an ioctl to trigger the deferred initialization (and memory free)
One of the main sub-systems that we defer initialization of this
way is USB, and this saves quite a bit of time. (Of course the
same, or slightly more CPU cycles are eventually used during
bootup time. But this lets us get to user space quicker so we
can start user-visible applications faster.)
I'm not that happy using an ioctl for this trigger. What is
the preferred method of activating a kernel feature like this?
I presume something in /proc or /sys, but I'm not sure.
Also, is there something like this in the kernel now that I'm
missing?
Any advice or comments would be welcome.
Thanks,
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Alexander Neundorf @ 2008-06-17 14:22 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080617134635.GE9141@nibiru.local>
On Tuesday 17 June 2008 15:46:36 Enrico Weigelt wrote:
> * Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> > On Monday 16 June 2008 17:15:37 Enrico Weigelt wrote:
> > > * Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> > > > CMake has a cache, where the values of variables are stored, e.g. if
> > > > an option is enabled or not, or where a library has been found (e.g.
> > > > JPEG_LIBRARY=/usr/local/lib/libjpeg.so).
> > > > The way to influence the behaviour of cmake is to change the value of
> > > > these variables, this can be done either via a GUI (curses based or
> > > > with cmake 2.6 also a graphical one), or via the command line:
> > > > $ cmake -D<name_of_variable>=<new_value> ...more options
> > >
> > > Are these variables strictly specified or is all left to individual
> > > author's decision ?
> >
> > Authors decision.
>
> Then you've got the same problem as w/ autoconf's config.status:
> You have to tweak it for each individual package separately :(
Well, these are all more or less scripting languages, so people can create
whichever variables they like, no way to enforce something.
Still one can suggest standards which should be followed.
Alex
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Enrico Weigelt @ 2008-06-17 14:11 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <200806151648.57679.rob@landley.net>
* Rob Landley <rob@landley.net> schrieb:
Hi,
> You can't get away from cross compiling whenever you want to bootstrap a new
> platform. But cross compiling can be minimized and encapsulated. It can be
> a stage you pass through to get it over with and no longer have to deal with
> it on the other side, which is the approach I take.
That's not enought for me. I want more encapsulation (yes, but buildfarm
is also running in a VZ, but still sysroot'ed). For example, I *want* certain
tests (like running target code) to fail, since I *do not* trust them.
(btw: I also let compiler warnings fail, just to be sure).
I, personally, prefer high build-time constraints, because runtime tests
are quite limited.
> By my count sysroot is the fifth layer of path logic the gcc guys have added
> in an attempt to paint over the dry rot.
I see this as a quite good separation between running and target system,
just one more fence for clear borders. As long as you don't pass absolute
search pathes to it, you can be sure that it doesn't mix up target and host.
Actually, I never want to crosscompile w/o it.
> Personally I use a derivative of the old uClibc wrapper script that rewrites
> the command line to start with "--nostdinc --nostdlib" and then builds it
> back up again without having any paths in there it shouldn't.
Might work, but then you're limited to some specific cases.
If you *only* intent bootstrapping of an minimal system, fine, but for
my projects too complex to handle.
> > #2: fix broken configure.in's (and feed back to upstream or OSS-QM)
>
> Whack-a-mole. Fun for the whole family. Only problem is, it never stops.
Most times, it as only to be done one per package. And it's an clean solution.
> > #3: replace libtool by unitool
>
> Uninstall libtool and don't replace it with anything, it's a NOP on Linux.
The problem is: many packages are entirely built upon this. So you'll have
to de-libtoolize them. Very much work. As I already had Unitool, I preferred
investing just a few hours for creating an generic drop-in replacement for
libtool instead of touching each single package.
> > Only crap sw looks at /proc at build time.
> > Yes, there's *much* crap sw out there :(
>
> 99% of all the developers out there don't really care about portability, and
> never will. Even if you eliminate the windows guys and the people who don't
> do C, 90% of the people who are _left_ get to work on the PC first, get it to
> work natively on other Linux platforms afterwards.
True :(
Some packages out there even don't have an clean native build path, eg.
requiring parts of the package already built and installed (-> firebird-db)
> Cross compiling is a step beyond "portability". They'll _never_ care about
> cross compiling. If they get inspired to make it work on MacOS X, then
> you'll have to extract the source and _build_ it on MacOS X to make that
> work. And 99% of all developers will nod their heads and go "quite right, as
> it should be".
>
> This isn't going to change any time soon.
Actually, I don't care much about that. I concentrate on getting those
packages I need cleaned-up and crosscompile'able - even if this means
going alone and maintaining an own branch.
If I sum up all the invested working hours of all the last years
on that and substract the total saved time from other projects
where I'm reusing this work, I get a positive balance ;-P
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Enrico Weigelt @ 2008-06-17 13:57 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080616173839.GA19341@cs181133002.pp.htv.fi>
* Adrian Bunk <bunk@kernel.org> schrieb:
Hi,
> > I won't to that w/ my TreeBuild. It is intentionally limited on
> > easily structured packages. People should either structure their
> > packages properly use something else ;-P
>
> For simple packages autoconf+automake+libtool is already near at your
> descriptive paragidm.
Yes, but mine is much simpler and syntactically stable. With autotools
there are uncountable things you can do wrong, without even noticing -
it needs *very* much care. In TreeBuild there's almost no room for
those mistakes. A simple typo can lead autoconf to unpredictable
behaviour - TreeBuild spits out an parse error.
So for these cases (which already make up a very large number of
total packages in the world), maintaining buildfiles becomes a lot
easier ;-P
> And despite all nasty details of autoconf/automake/libtool they also
> have advantages:
> - they are quite powerful when you know how to handle them
The point is: you *must* know autotools very well, otherwise you
soon run into pitfalls.
> - they allow to build your software on non-Linux systems
TreeBuild does that too, as soon as someone wrote a proper config
for another platform. The design is platform agnostic, just lacking
ports to other platforms, due lack of (my) time.
> - they are the de-facto standard in the open source world, and everyone
> building open source software knows them
I don't think that this is a good argument. M$-Word is also an
"de facto" standard ;-P
> And the last point is a very important one:
>
> For me as someone who is often compiling software your plan of creating
> yet another build tool I have to handle does not sound like a good idea.
Yes, but that's not a problem of TreeBuild, but the total count of
different build systems around the world. That's why a new buildsystem
should be an really good solution for an reasonable class of problems
and should be easy to learn. IMHO, TreeBuild does this - for simple
structed packages. (which make up about 99% of my own ones).
Of course, TreeBuild still in an early development stage. So I
don't claim it's really usable for everyone - for now it's more
an academic issue.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Enrico Weigelt @ 2008-06-17 13:46 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <200806170827.09103.neundorf@eit.uni-kl.de>
* Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> On Monday 16 June 2008 17:15:37 Enrico Weigelt wrote:
> > * Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> > > CMake has a cache, where the values of variables are stored, e.g. if an
> > > option is enabled or not, or where a library has been found (e.g.
> > > JPEG_LIBRARY=/usr/local/lib/libjpeg.so).
> > > The way to influence the behaviour of cmake is to change the value of
> > > these variables, this can be done either via a GUI (curses based or with
> > > cmake 2.6 also a graphical one), or via the command line:
> > > $ cmake -D<name_of_variable>=<new_value> ...more options
> >
> > Are these variables strictly specified or is all left to individual
> > author's decision ?
>
> Authors decision.
Then you've got the same problem as w/ autoconf's config.status:
You have to tweak it for each individual package separately :(
My destiny is to have strictly standardized variables for all the
common things, so you can use an global per-target configuration
for *all* packages ever coming.
That's what Unitool's system properties db is for.
> > hmm, why not just expecting an sane shell on the building system ?
> > (as you already have to expect a sane compiler)
>
> Well, we could go so far to expect a "sane" operating system, but you can't
> change it, there are "insane" people in the world ;-)
Many, many things can be done within in the toolchain, eg. fixes
for broken libc's. For example, I've seen packages adding several
missing functions for certain platforms - this should be the job
of the toolchain.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Alexander Neundorf @ 2008-06-17 6:27 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080616151536.GC9141@nibiru.local>
On Monday 16 June 2008 17:15:37 Enrico Weigelt wrote:
> * Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> > CMake has a cache, where the values of variables are stored, e.g. if an
> > option is enabled or not, or where a library has been found (e.g.
> > JPEG_LIBRARY=/usr/local/lib/libjpeg.so).
> > The way to influence the behaviour of cmake is to change the value of
> > these variables, this can be done either via a GUI (curses based or with
> > cmake 2.6 also a graphical one), or via the command line:
> > $ cmake -D<name_of_variable>=<new_value> ...more options
>
> Are these variables strictly specified or is all left to individual
> author's decision ?
Authors decision.
> > > autotools need only a shell and make
> >
> > Yes, and requiring a shell is a lot, in the case you want to port
> > also to Windows.
>
> hmm, why not just expecting an sane shell on the building system ?
> (as you already have to expect a sane compiler)
Well, we could go so far to expect a "sane" operating system, but you can't
change it, there are "insane" people in the world ;-)
Alex
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Adrian Bunk @ 2008-06-16 22:44 UTC (permalink / raw)
To: Jamie Lokier
Cc: Bernhard Fischer, Alexander Neundorf, Linux Embedded Maillist
In-Reply-To: <20080616222835.GA29899@shareable.org>
On Mon, Jun 16, 2008 at 11:28:36PM +0100, Jamie Lokier wrote:
> Bernhard Fischer wrote:
> > On Mon, Jun 16, 2008 at 02:32:01PM +0100, Jamie Lokier wrote:
> >
> > >No, I'm talking about improving Autotools to handle some things better
> > >than they do now. Passing the high hurdles required to become part of
> > >Autotools - especially compatibility - is part of the goal.
> >
> > If you look at the sh scripts generated by autotools one is tempted to
> > just ship a small, clean sh, written in C89 with autotools, use that
> > and drop the workarounds.
>
> Exactly!
>
> Use an early, small Autoconf script written in very portable shell (as
> now) to detect the small set of features needed for this "clean sh",
> and a host compiler. Then build it, then continue in clean sh.
>
> But if you're doing that, why use sh? It could be a subset of sh with
> extensions especially for this job - or something else entirely - to
> make Autoconf scripts cleaner, smaller, faster than they are now. Or
> even a tokenised sh, so Autoconf tests run directly on it but are
> compact. Possibly even _more_ portable (no need to depend on quirks
> of some utilities; configure scripts more checkable for common
> errors). And easier to configure things on environments with no shell
> (i.e. Windows).
>
> It adds an extra requirement for a host C compiler, but that's
> probably reasonable these days.
>
> This conversation ought to be heading over to the autotools
> lists... if anyone wants to take it up there.
All public ramblings about how autoconf might be improved don't bring
anything if you do not implement your ideas yourself and show that your
implementation is better than the status quo.
After all, open source development is not driven by people producing hot
air but by people writing code...
> -- Jamie
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 22:28 UTC (permalink / raw)
To: Bernhard Fischer; +Cc: Alexander Neundorf, Linux Embedded Maillist
In-Reply-To: <20080616162811.GA22089@mx.loc>
Bernhard Fischer wrote:
> On Mon, Jun 16, 2008 at 02:32:01PM +0100, Jamie Lokier wrote:
>
> >No, I'm talking about improving Autotools to handle some things better
> >than they do now. Passing the high hurdles required to become part of
> >Autotools - especially compatibility - is part of the goal.
>
> If you look at the sh scripts generated by autotools one is tempted to
> just ship a small, clean sh, written in C89 with autotools, use that
> and drop the workarounds.
Exactly!
Use an early, small Autoconf script written in very portable shell (as
now) to detect the small set of features needed for this "clean sh",
and a host compiler. Then build it, then continue in clean sh.
But if you're doing that, why use sh? It could be a subset of sh with
extensions especially for this job - or something else entirely - to
make Autoconf scripts cleaner, smaller, faster than they are now. Or
even a tokenised sh, so Autoconf tests run directly on it but are
compact. Possibly even _more_ portable (no need to depend on quirks
of some utilities; configure scripts more checkable for common
errors). And easier to configure things on environments with no shell
(i.e. Windows).
It adds an extra requirement for a host C compiler, but that's
probably reasonable these days.
This conversation ought to be heading over to the autotools
lists... if anyone wants to take it up there.
-- Jamie
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Adrian Bunk @ 2008-06-16 17:38 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080616160037.GD9141@nibiru.local>
On Mon, Jun 16, 2008 at 06:00:37PM +0200, Enrico Weigelt wrote:
> * Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
>
> > > That's why I prefer an purely descriptive paragidm (= subset of
> > > delcarative, but practically no logic): a buildfile should only
> > > describe the package's structure (eg.: "i have some executable foo
> > > which consists of source [...] and imports libs [...]), so the
> > > buildtool (and user's config) can cope with it all.
> >
> > This was also the plan for cmake when it was started. But it turned
> > out that this is not enough, and for more complex projects some
> > programming logic is required. So cmake turned from quite declarative
> > to quite imperative over time.
>
> And so you just open the Pandorra's Box again ;-o
> As soon as you try to build some one-fits-all-solution, you'll
> sooner or later run into similar problems as autoconf has.
>
> I won't to that w/ my TreeBuild. It is intentionally limited on
> easily structured packages. People should either structure their
> packages properly use something else ;-P
For simple packages autoconf+automake+libtool is already near at your
descriptive paragidm.
And despite all nasty details of autoconf/automake/libtool they also
have advantages:
- they are quite powerful when you know how to handle them
- they allow to build your software on non-Linux systems
- they are the de-facto standard in the open source world, and everyone
building open source software knows them
And the last point is a very important one:
For me as someone who is often compiling software your plan of creating
yet another build tool I have to handle does not sound like a good idea.
> cu
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply
* Re: cross-compiling alternatives
From: Bernhard Fischer @ 2008-06-16 16:45 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080616143315.GB9141@nibiru.local>
On Mon, Jun 16, 2008 at 04:33:15PM +0200, Enrico Weigelt wrote:
>* Peter Korsgaard <jacmet@sunsite.dk> schrieb:
>
>Hi,
>
>> Notice that sysroot is broken in 0.23 (it removes everything else than
>> -I / -L words). Patches have been posted to the pkgconfig list several
>> times.
heh, so it keeps e.g. -I=/usr/include ? Fun ;)
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bernhard Fischer @ 2008-06-16 16:43 UTC (permalink / raw)
To: Jamie Lokier; +Cc: David Woodhouse, Enrico Weigelt, linux-embedded
In-Reply-To: <20080616115218.GF18857@shareable.org>
On Mon, Jun 16, 2008 at 12:52:18PM +0100, Jamie Lokier wrote:
>The same applications are fine if pread returns ENOSYS and they know
>what they need to do with lseek and read.
Note that they (pread, etc.) would better not exist at all instead of
being stubbed out (this requirement was dropped, AFAICS). If there is no
prototype then you don't need AC_TRY_RUN(other_arch_pread_test) in the
first place. Of course this assumes that you don't want to make that
decision at runtime but compile time.
>
>The same is true of the rsync example, with utimes and utime. It's
>wrong for libc to "emulate" utimes using utime on old kernels: they
>aren't the same. So the application does it instead: the application
>has a policy that it's fine with the different functionality when
>utimes returns ENOSYS.
You usually do the opposite, emulate utimes with utime ;)
link_warning(utimes, "the use of LEGACY `utimes' is discouraged, use `utime'")
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Bernhard Fischer @ 2008-06-16 16:28 UTC (permalink / raw)
To: Jamie Lokier; +Cc: Alexander Neundorf, Linux Embedded Maillist
In-Reply-To: <20080616133201.GG18857@shareable.org>
On Mon, Jun 16, 2008 at 02:32:01PM +0100, Jamie Lokier wrote:
>No, I'm talking about improving Autotools to handle some things better
>than they do now. Passing the high hurdles required to become part of
>Autotools - especially compatibility - is part of the goal.
If you look at the sh scripts generated by autotools one is tempted to
just ship a small, clean sh, written in C89 with autotools, use that
and drop the workarounds.
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Enrico Weigelt @ 2008-06-16 16:00 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <200806160958.48157.neundorf@eit.uni-kl.de>
* Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> > That's why I prefer an purely descriptive paragidm (= subset of
> > delcarative, but practically no logic): a buildfile should only
> > describe the package's structure (eg.: "i have some executable foo
> > which consists of source [...] and imports libs [...]), so the
> > buildtool (and user's config) can cope with it all.
>
> This was also the plan for cmake when it was started. But it turned
> out that this is not enough, and for more complex projects some
> programming logic is required. So cmake turned from quite declarative
> to quite imperative over time.
And so you just open the Pandorra's Box again ;-o
As soon as you try to build some one-fits-all-solution, you'll
sooner or later run into similar problems as autoconf has.
I won't to that w/ my TreeBuild. It is intentionally limited on
easily structured packages. People should either structure their
packages properly use something else ;-P
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Enrico Weigelt @ 2008-06-16 15:15 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <200806160955.12752.neundorf@eit.uni-kl.de>
* Alexander Neundorf <neundorf@eit.uni-kl.de> schrieb:
> CMake has a cache, where the values of variables are stored, e.g. if an option
> is enabled or not, or where a library has been found (e.g.
> JPEG_LIBRARY=/usr/local/lib/libjpeg.so).
> The way to influence the behaviour of cmake is to change the value of these
> variables, this can be done either via a GUI (curses based or with cmake 2.6
> also a graphical one), or via the command line:
> $ cmake -D<name_of_variable>=<new_value> ...more options
Are these variables strictly specified or is all left to individual
author's decision ?
> > autotools need only a shell and make
>
> Yes, and requiring a shell is a lot, in the case you want to port
> also to Windows.
hmm, why not just expecting an sane shell on the building system ?
(as you already have to expect a sane compiler)
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives
From: Enrico Weigelt @ 2008-06-16 14:33 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <87zlplj0ot.fsf@macbook.be.48ers.dk>
* Peter Korsgaard <jacmet@sunsite.dk> schrieb:
Hi,
> Notice that sysroot is broken in 0.23 (it removes everything else than
> -I / -L words). Patches have been posted to the pkgconfig list several
> times.
ugh, why didn't they just apply my patch which only appends
$sysroot to the pathes ? .... grmpf ...
well, i'll stick w/ my 5-lines wrapper script ;-o
> Enrico> Yep, it doesn't take $SYSROOT (as my original patch did), but with
> Enrico> some prefix ...
>
> PKG_CONFIG_SYSROOT_DIR
ack.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 13:32 UTC (permalink / raw)
To: Alexander Neundorf; +Cc: Linux Embedded Maillist
In-Reply-To: <200806161406.40063.neundorf@eit.uni-kl.de>
Alexander Neundorf wrote:
> On Monday 16 June 2008 13:39:46 you wrote:
> ...
> > > > If you're going to rewrite Autotools using GNU Make, why not ask if
> > > > another tool would be better, perhaps a tool specially designed for
> > > > the job?
> > >
> > > Go ahead.
> >
> > The first part of going ahead is to brainstorm and find ideas and
> > likely interest... It's too big (as you rightly note) to simply sit
> > down and do it by oneself in isolation.
>
> You are not seriously talking about writing yet another buildsystem?
> With autotools, cmake, scons, ant, (b)jam, bitbake, makeng,
> etc. there should be enough available.
No, I'm talking about improving Autotools to handle some things better
than they do now. Passing the high hurdles required to become part of
Autotools - especially compatibility - is part of the goal.
-- Jamie
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Alexander Neundorf @ 2008-06-16 12:06 UTC (permalink / raw)
To: Linux Embedded Maillist
In-Reply-To: <20080616113946.GD18857@shareable.org>
On Monday 16 June 2008 13:39:46 you wrote:
...
> > > If you're going to rewrite Autotools using GNU Make, why not ask if
> > > another tool would be better, perhaps a tool specially designed for
> > > the job?
> >
> > Go ahead.
>
> The first part of going ahead is to brainstorm and find ideas and
> likely interest... It's too big (as you rightly note) to simply sit
> down and do it by oneself in isolation.
You are not seriously talking about writing yet another buildsystem ?
With autotools, cmake, scons, ant, (b)jam, bitbake, makeng, etc. there should
be enough available.
Alex
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: David Woodhouse @ 2008-06-16 11:59 UTC (permalink / raw)
To: Jamie Lokier; +Cc: Enrico Weigelt, linux-embedded
In-Reply-To: <20080616115218.GF18857@shareable.org>
On Mon, 2008-06-16 at 12:52 +0100, Jamie Lokier wrote:
> E.g. Calls to pread should _not_ be implemented as lseek+read+lseek on
> old kernels which don't have pread. That leads to race conditions and
> corruption in some applications. (I think this has really occurred,
> but I'm unable to find it now).
Likewise pselect(). You're right -- it can't always be emulated.
--
dwmw2
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 11:52 UTC (permalink / raw)
To: David Woodhouse; +Cc: Enrico Weigelt, linux-embedded
In-Reply-To: <1213614593.26255.691.camel@pmac.infradead.org>
David Woodhouse wrote:
> On Mon, 2008-06-16 at 11:49 +0100, Jamie Lokier wrote:
> > But here's the thing: do you really want every package have code
> > calling every different variation on a system call, at run time, until
> > it finds one that works?
>
> No. That functionality lives in libc, if you want it at all.
Sometimes that's ok.
But sometimes libc can't do that, because the point of a newer
function is it addresses some inadequacy of an old function. If libc
does a "compatibility" thing and uses old syscalls on old kernels,
sometimes that introduces bugs.
E.g. Calls to pread should _not_ be implemented as lseek+read+lseek on
old kernels which don't have pread. That leads to race conditions and
corruption in some applications. (I think this has really occurred,
but I'm unable to find it now).
The same applications are fine if pread returns ENOSYS and they know
what they need to do with lseek and read.
The same is true of the rsync example, with utimes and utime. It's
wrong for libc to "emulate" utimes using utime on old kernels: they
aren't the same. So the application does it instead: the application
has a policy that it's fine with the different functionality when
utimes returns ENOSYS.
-- Jamie
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 11:44 UTC (permalink / raw)
To: linux-embedded
In-Reply-To: <20080616045715.GC32139@nibiru.local>
Enrico Weigelt wrote:
> > Imho, Kconfig would be good for more programs than it's currently used for,
> > and could be made to work with those --enable/--with options: you'd be
> > able to configure them entirely on the command line, or interactively
> > with "./configure --menu" (runs menuconfig), or with a config file.
>
> Yes, that would be fine. But for me the primary constraint is that
> all switches/options can be specified by command line - otherwise
> I'd need extra complexity for each package in my distbuilder tool.
Yes, I agree with that. It's one of the nice things about Kconfig
that you can use it with pre-specific config files, even when the
config is from a different version and doesn't specify everything.
Doing so as ./configure --enable/--disable arguments seems like a
comfortable fit.
>
> > Perhaps it might even be possible to write a very small, portable,
> > specialised alternative to Make which is small enough to ship with
> > packages that use it?
>
> No, I really wouldn't advise this. Make tools are, IMHO, part of
> the toolchain (in a wider sense). Once point is avoiding code
> duplication, but the really important one is: a central point of
> adaption/configuration. That's eg. why I like pkg-config so much:
> if I need some tweaking, I just pass my own command (or a wrapper).
> If each package does it's library lookup completely by itself, I
> also need to touch each single package in case I need some tweaks.
> I had exactly that trouble w/ lots of packages, before I ported
> them to pkg-config.
That's interesting, thanks. I have seen problems with pkg-config, but
adding your own wrapper is a nice way to fix any of them :-)
That said, if you can specify library lookup on the ./configure
command line, that fixes a lot of problems of repeatability too.
Only problem then is you don't always know _which_ options to pass on
the ./configure command line, if you have a package build script, and
a different version of the package. There's a tendancy to drift apart
which manifests as silent mistakes, until some user reports a bug.
-- Jamie
^ permalink raw reply
* Re: cross-compiling alternatives
From: Bernd Petrovitsch @ 2008-06-16 11:43 UTC (permalink / raw)
To: Jamie Lokier; +Cc: Robert Schwebel, Alexander Neundorf, linux-embedded
In-Reply-To: <20080616111752.GB18857@shareable.org>
On Mon, 2008-06-16 at 12:17 +0100, Jamie Lokier wrote:
> Bernd Petrovitsch wrote:
> > > _check_ for many installed libraries. Get them wrong, and you have
> > > the same problems of untested combinations.
> >
> > As long as I can specify that libfoo support must be compiled in (and
> > thus libfoo must be present) and the tools throw an error if it doesn't
> > find it, I have no problem.
> > Otherwise all package builders have a serious problem.
>
> They do have problems, when you want to repeatably build and deploy,
> if the build environment isn't very similar each time.
Sometimes you have a different build environments - if only that you
want to rebuild e.g. your .src.rpm on several versions of CentOS and
Fedora.
> Typically the way you specify that libfoo support must be compiled in
> is --with-libfoo=/path/to/libfoo.
>
> That way lies bitrot between your build script which calls ./configure
Cannot be really avoided IMHO.
> (since you won't by typing it manually with 20+ options like that each
> time you rebuild), and the changing version of an upstream package you
> configure.
So be it. At least one sees errors/bugs immediately.
> To prevent it trying to compile in libs you don't want, you also need
> --without-libfoo. Using Kerberos as an example, which I remember when
> building CVS ages ago: If you don't _prevent_ it using libraries you
> don't want, you get different binariesn depending on whether a
> Kerberos library was installed on the build system at build time. You
> might then send a built program to another system, and find it won't
> run at all, or has unwanted behaviour.
>
> Do you really see package building scripts with 20 --with-libfoo= and
> --without-libfoo= options in them for every library? Sometimes. But
For (in an ideal world 100%) repeatable builds - for a .rpm. .deb, some
cross-compiled embedded device - one usually ends up with that explicit
list (and IMHO it's the smaller PITA than to cope with strange bug
reports because something was broken in the build weeks ago).
Mainly because the dependency information is also present elsewehre
(e.g. in the package). Or you really want control over the installed
software.
> more often, not: instead, they more often have build-time installed
> prerequisites.
Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 11:39 UTC (permalink / raw)
To: Robert Schwebel; +Cc: Alexander Neundorf, linux-embedded
In-Reply-To: <20080614112656.GX13599@pengutronix.de>
Robert Schwebel wrote:
> We have once tried to write our automated test system for embedded
> boards with python and xml; the idea was to have something fancy, new
> and with good code-reuse. In the end it failed because the pexpect
> package we used to do the pattern matching bitrotted so quickly that
> four months later all the fancy tests didn't work any more, because it
> is not part of the python core.
>
> In the meantime we have migrated our automatic testing stuff to use
> shell, ssh/rsh and kermit. It is rock solid, and the code reuse factor
> is at least as good as with anything else.
It sounds like the basic problem there was using non-core Python. You
can count on shell, ssh/rsh, kernel - they're stable. You can count
on Pythong's core language. But not on pexpect: if you depended on
that, you should probably have copied the code into your own test
system, or written an equivalent. Easier said than done, I know.
> > Have you felt uncomfortable shipping a package that does use Autoconf,
> > Automake and Libtool, knowing that the scripts generated by those
> > tools are huge compared with the entire source of your package?
>
> No :-)
It's really disturbed me, sometimes, shipping a package of 50kB source
and 2MB configure+libtool.
> > Have you _written_ Autoconf tests recently?
>
> Yea, all our packages are autotoolized.
>
> > Made any shell / shellutils non-portability mistakes in the tests?
>
> Yea, it happens in ptxdist all the time. People report about problems,
> we add new tests and the next revision works even on Ubuntu :)
Right, so you have no idea about portability of your tests on, say, SCO
Unixserver or IRIX? :-)
> I have no problems with people writing fancy new things. It's just that
> most people who try to do something better than autotools have only a
> fraction of the features.
>
> Open Source is darwinism: if there is something better, let's use it.
> But compare apples with apples.
>
> > If you're going to rewrite Autotools using GNU Make, why not ask if
> > another tool would be better, perhaps a tool specially designed for
> > the job?
>
> Go ahead.
The first part of going ahead is to brainstorm and find ideas and
likely interest... It's too big (as you rightly note) to simply sit
down and do it by oneself in isolation.
-- Jamie
^ permalink raw reply
* Re: cross-compiling alternatives (was Re: [PATCH 0/1] Embedded Maintainer(s)...)
From: Jamie Lokier @ 2008-06-16 11:33 UTC (permalink / raw)
To: linux-embedded
In-Reply-To: <20080616051147.GD32139@nibiru.local>
Enrico Weigelt wrote:
> > Reality is that Kconfig front end to autotools does work - as you've
> > proved. It's a good idea. :-)
>
> Now, we just need an autoconf-alike frontend for Kconfig ;-)
I agree and would support:
./configure --menu
Invokes a configuration menu / wizard for user-selectable options.
Only works if you have the "autoconf-menu" tools already installed.
./configure --ask
Invokes command line configuration questions / wizard for
user-selectable options.
./configure --ask --from-config=OLDCONFIG
Uses existing settings from a previous configuration.
./configure
Without --menu or --ask, invokes traditional Autoconf
automatic detection of capabilities.
> > You said about too many user-selectable options. Many large packages
> > _check_ for many installed libraries. Get them wrong, and you have
> > the same problems of untested combinations.
>
> It even gets worse when they silently enable certain features on
> presence/absence of some lib. That's btw one of the reasons why
> sysroot is an primary constraint for me, even when building for the
> platform+arch.
That's a basic design feature of Autoconf. And it really is very good
most of the time - much better than config scripts which ask a lot of
questions (Perl).
> > Have you felt uncomfortable shipping a package that does use Autoconf,
> > Automake and Libtool, knowing that the scripts generated by those
> > tools are huge compared with the entire source of your package?
>
> Yeah, that's one of those things in autotools I never understood:
> why isn't there just one function for each type of check/action,
> which is just called with the right params ?
It's because it's written in very portable Bourne shell, which does
not support shell functions.
(This is why I toy with the idea of writing a very portable C program,
similar to a simplified shell, specially designed for Autoconf, with
functions...)
It could simulate functions portably by creating lots of little
scripts at the start, and then calling them later on. That would
shrink ./configure a lot. But it's a big change; somebody's got to do
it, test it, look after it, and so on.
Libtool similarly does not have to be huge, but it would be a lot of
work to shrink it without breaking anything. It's hard enough to
change it at all without breaking anything on some system, somewhere.
-- Jamie
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox