* Re: mainlining min-configs...
From: Sam Ravnborg @ 2008-06-11 19:48 UTC (permalink / raw)
To: Adrian Bunk
Cc: Tim Bird, Rob Landley, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080611193639.GB2958@cs181133002.pp.htv.fi>
On Wed, Jun 11, 2008 at 10:36:39PM +0300, Adrian Bunk wrote:
> On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> > Sam Ravnborg wrote:
> > >>> When I get my kconfig patchset polished you will be able to do:
> > >>>
> > >>> make K=my_mini_config allnoconfig
> > >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > >
> > > Somehow yes. The K= notation I hope will see more use. Only very few
> > > people know the KCONFIG_ALL*CONFIG= trick.
> >
> > Indeed. I had never heard of it, but it appears to have been
> > around for at least 18 months. It looks like Randy Dunlap tried
> > to submit some documentation for this in Oct 2006, but it didn't
> > make it in?
> >...
>
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.
The one I refer to is not.
Sam
^ permalink raw reply
* Re: mainlining min-configs...
From: Tim Bird @ 2008-06-11 19:46 UTC (permalink / raw)
To: Adrian Bunk
Cc: Sam Ravnborg, Rob Landley, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <20080611193639.GB2958@cs181133002.pp.htv.fi>
Adrian Bunk wrote:
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.
Well, dangit there it is!
The patch I googled had it going into Documentation/kbuild. It
somehow escaped my attention in the README. If I was
a little more skilled with my grep-ing I would have found it.
Sorry for the bother!
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: mainlining min-configs...
From: Adrian Bunk @ 2008-06-11 19:36 UTC (permalink / raw)
To: Tim Bird
Cc: Sam Ravnborg, Rob Landley, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <48502304.1030304@am.sony.com>
On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> >
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
>
> Indeed. I had never heard of it, but it appears to have been
> around for at least 18 months. It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
>...
Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
April 2006.
> -- Tim
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply
* Re: mainlining min-configs...
From: Sam Ravnborg @ 2008-06-11 19:22 UTC (permalink / raw)
To: Tim Bird; +Cc: Rob Landley, Adrian Bunk, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <48502304.1030304@am.sony.com>
On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> >
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
>
> Indeed. I had never heard of it, but it appears to have been
> around for at least 18 months. It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
>
> Is this feature widely used? Should it be doc'ed now
> or are you about to make it obsolete?
I will most likely obsolete all except KCONFIG_ALLCONFIG
And I'm afraid I have Randy patch somewhere.
Sam
^ permalink raw reply
* Re: mainlining min-configs...
From: Tim Bird @ 2008-06-11 19:09 UTC (permalink / raw)
To: Sam Ravnborg
Cc: Rob Landley, Adrian Bunk, linux-tiny, linux-embedded,
linux kernel
In-Reply-To: <20080611063921.GA27978@uranus.ravnborg.org>
Sam Ravnborg wrote:
>>> When I get my kconfig patchset polished you will be able to do:
>>>
>>> make K=my_mini_config allnoconfig
>> So you're renaming KCONFIG_ALLNOCONFIG then?
>
> Somehow yes. The K= notation I hope will see more use. Only very few
> people know the KCONFIG_ALL*CONFIG= trick.
Indeed. I had never heard of it, but it appears to have been
around for at least 18 months. It looks like Randy Dunlap tried
to submit some documentation for this in Oct 2006, but it didn't
make it in?
Is this feature widely used? Should it be doc'ed now
or are you about to make it obsolete?
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Tim Bird @ 2008-06-11 18:23 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Sam Ravnborg, linux-embedded, linux kernel, David Woodhouse,
Holger Schurig
In-Reply-To: <Pine.LNX.4.64.0806111328150.13310@vixen.sonytel.be>
Geert Uytterhoeven wrote:
> No checking for excess arguments?
Holger Schurig wrote:
> Would it be helpful to compare .config.old to .config if you
> don't provide any command line arguments?
Both good ideas. These are implemented in the latest version.
Note that I check for and use KBUILD_OUTPUT. (I always put
my build output outside the source directory, since
I usually build for multiple arches from a single tree.)
The program is also now better structured, IMHO.
-- Tim
Diffconfig is a simple utility for comparing two .config files.
See usage in the script for more info.
Signed-off-by: Tim Bird <tim.bird@am.sony.com>
scripts/diffconfig | 128 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 128 insertions(+)
create mode 100755 scripts/diffconfig
diff --git a/scripts/diffconfig b/scripts/diffconfig
new file mode 100755
index 0000000..aa6cfe1
--- /dev/null
+++ b/scripts/diffconfig
@@ -0,0 +1,128 @@
+#!/usr/bin/python
+#
+# diffconfig - a tool to compare .config files.
+#
+# originally written in 2006 by Matt Mackall
+# (at least, this was in his bloatwatch source code)
+# last worked on 2008 by Tim Bird
+#
+
+import sys, os
+
+def usage():
+ print "Usage: diffconfig [-h] [-m] [<config1> <config2>]\n"
+ print """Diffconfig is a simple utility for comparing two .config files.
+Using standard diff to compare .config files often includes extraneous and
+distracting information. This utility produces sorted output with only the
+changes in configuration values between the two files.
+
+Added and removed items are shown with a leading plus or minus, respectively.
+Changed items show the old and new values on a single line.
+
+If -m is specified, then output will be in "merge" style, which has the changed
+and new values in kernel config option format.
+
+If no config files are specified, .config and .config.old are used.
+
+Example usage:
+ $ diffconfig .config config-with-some-changes
+-EXT2_FS_XATTR n
+-EXT2_FS_XIP n
+ CRAMFS n -> y
+ EXT2_FS y -> n
+ LOG_BUF_SHIFT 14 -> 16
+ PRINTK_TIME n -> y
+"""
+ sys.exit(0)
+
+# returns a dictionary of name/value pairs for config items in the file
+def readconfig(config_file):
+ d = {}
+ for line in config_file:
+ line = line[:-1]
+ if line[:7] == "CONFIG_":
+ name, val = line[7:].split("=", 1)
+ d[name] = val
+ if line[-11:] == " is not set":
+ d[line[9:-11]] = "n"
+ return d
+
+def print_config(op, config, value, new_value):
+ global merge_style
+
+ if merge_style:
+ if new_value:
+ if new_value=="n":
+ print "# CONFIG_%s is not set" % config
+ else:
+ print "CONFIG_%s=%s" % (config, new_value)
+ else:
+ if op=="-":
+ print "-%s %s" % (config, value)
+ elif op=="+":
+ print "+%s %s" % (config, new_value)
+ else:
+ print " %s %s -> %s" % (config, value, new_value)
+
+def main():
+ global merge_style
+
+ # parse command line args
+ if ("-h" in sys.argv or "--help" in sys.argv):
+ usage()
+
+ merge_style = 0
+ if "-m" in sys.argv:
+ merge_style = 1
+ sys.argv.remove("-m")
+
+ argc = len(sys.argv)
+ if not (argc==1 or argc == 3):
+ print "Error: incorrect number of arguments or unrecognized option"
+ usage()
+
+ if argc == 1:
+ # if no filenames given, assume .config and .config.old
+ build_dir=""
+ if os.environ.has_key("KBUILD_OUTPUT"):
+ build_dir = os.environ["KBUILD_OUTPUT"]+"/"
+
+ configa_filename = build_dir + ".config.old"
+ configb_filename = build_dir + ".config"
+ else:
+ configa_filename = sys.argv[1]
+ configb_filename = sys.argv[2]
+
+ a = readconfig(file(configa_filename))
+ b = readconfig(file(configb_filename))
+
+ # print items in a but not b (accumulate, sort and print)
+ old = []
+ for config in a:
+ if config not in b:
+ old.append(config)
+ old.sort()
+ for config in old:
+ print_config("-", config, a[config], None)
+ del a[config]
+
+ # print items that changed (accumulate, sort, and print)
+ changed = []
+ for config in a:
+ if a[config] != b[config]:
+ changed.append(config)
+ else:
+ del b[config]
+ changed.sort()
+ for config in changed:
+ print_config("->", config, a[config], b[config])
+ del b[config]
+
+ # now print items in b but not in a
+ # (items from b that were in a were removed above)
+ new = b.keys()
+ new.sort()
+ for config in new:
+ print_config("+", config, None, b[config])
+
+main()
^ permalink raw reply related
* Re: [PATCH] add diffconfig utility
From: Geert Uytterhoeven @ 2008-06-11 11:28 UTC (permalink / raw)
To: Tim Bird; +Cc: Sam Ravnborg, linux-embedded, linux kernel, David Woodhouse
In-Reply-To: <484F1B8F.9000804@am.sony.com>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 778 bytes --]
On Tue, 10 Jun 2008, Tim Bird wrote:
> +merge_style = 0
> +if "-m" in sys.argv:
> + merge_style = 1
> + sys.argv.remove("-m")
> +
> +if "-h" in sys.argv or len(sys.argv) < 3:
^^^
No checking for excess arguments?
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
^ permalink raw reply
* Re: mainlining min-configs...
From: Christian MICHON @ 2008-06-11 8:59 UTC (permalink / raw)
To: Tim Bird; +Cc: Rob Landley, linux-tiny, linux kernel, linux-embedded,
Adrian Bunk
In-Reply-To: <484DDAD9.2000306@am.sony.com>
On Tue, Jun 10, 2008 at 3:37 AM, Tim Bird <tim.bird@am.sony.com> wrote:
> Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> Does anyone else think this would be valuable? If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.
> -- Tim
>
this is very valuable. DetaolB uses this concept. Download the latest
iso of DetaolB, boot it and look for /proc/miniconfig.gz.
you'll see an example for 2.6.23 there.
you can find old patches there:
http://downloads.sourceforge.net/detaolb/detaolb_v03-linux-2.6.21.patch
--
Christian
--
http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !
^ permalink raw reply
* Re: [PATCH] console - Add configurable support for console charset translation
From: Holger Schurig @ 2008-06-11 7:08 UTC (permalink / raw)
To: Adrian Bunk; +Cc: Tim Bird, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080604103353.GC27335@cs181133002.pp.htv.fi>
> Does the linux-tiny approach of adding a kconfig variable for
> each 5kB of code actually make sense? I'm asking since an
> exploding amount of kconfig variables and their
> interdependencies have a not so small maintainance impact in
> the long term.
I don't want to answer for the general case, but I can answer for
my specific case.
My device has Intel Strataflash, which have 256 kB size of
erase-sectors. I reserved one sector for u-boot (which is
plenty) and 4 for linux --- which uses to be plenty in the
2.4.21 days.
It is no longer plenty, some years ago I switched one of the
targets to 2.6.15. The 4 sectors still were ok. Some months ago
I switched to 2.6.24/2.6.25 and now space is VERY scarce. Just
yesterday, when I trashed unionfs because of some misbehavior I
couldn't fix by myself and went with aufs. Now my kernel
suddenly became 14 kB too big for my device.
Now, tiny-linux patches are at 2.6.23, but I could still adapt a
bunch of them to 2.6.25 and with that and some changed configs
my headroom is now again 26460 bytes. Unfortunately, my custom
boot logo had to go :-/
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Holger Schurig @ 2008-06-11 6:59 UTC (permalink / raw)
To: Tim Bird; +Cc: linux-embedded, linux kernel
In-Reply-To: <484ED902.9040000@am.sony.com>
Nice.
Would it be helpful to compare .config.old to .config if you
don't provide any command line arguments?
scripts/diffconfig
would then do the job.
^ permalink raw reply
* Re: mainlining min-configs...
From: Sam Ravnborg @ 2008-06-11 6:39 UTC (permalink / raw)
To: Rob Landley
Cc: Adrian Bunk, Tim Bird, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <200806110009.22542.rob@landley.net>
> >
> > When I get my kconfig patchset polished you will be able to do:
> >
> > make K=my_mini_config allnoconfig
>
> So you're renaming KCONFIG_ALLNOCONFIG then?
Somehow yes. The K= notation I hope will see more use. Only very few
people know the KCONFIG_ALL*CONFIG= trick.
>
> > Thus selecting 'no' for all new symbols in an automated fashion.
> > I know that in a few cases 'no' is the wrong answer but in the
> > 99% of the cases 'no' is perfectly valid.
>
> For a "make miniconfig", warnings should be errors. (Attempts to set unknown
> symbols are an error with a miniconfig, the operation should exit with a
> nonzero error code.) Also, all the stuff allnoconfig puts to stdout
> shouldn't be there for miniconfig, only the stuff it writes to stderr right
> now should show up.
My reference was to a minimal config of a system - not to a miniconfig
as your patcset produces. Just to make it clear.
The patchset I refer will also reduce the nosie generated during make *config.
Sam
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-11 5:51 UTC (permalink / raw)
To: Tim Bird; +Cc: Adrian Bunk, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <484EC576.5080302@am.sony.com>
On Tuesday 10 June 2008 13:18:30 Tim Bird wrote:
> bitrot in mainline. My experience is that usually a 'make oldconfig'
> will produce something usable. But maybe that wouldn't be as
> effective with a minconfig?
I'm probably going to have to start breaking down and patching the kconfig
infrastructure. (Today, I drove allnoconfig into an endless loop. Go me.)
So far the only "gotcha" I've found is added guard symbols, and it's
semantically pretty clear what a miniconfig should do there: force open the
menu that contains a symbol you're setting, rather than _ignore_ that symbol.
> Maybe I'll collect some minconfigs, and try maintaining them
> in my own tree for a few releases to see how onerous it is...
The thing about my .configs is that I boot test them each new kernel release,
using qemu. This is 100% scriptable, actually, which you can seldom say
about real hardware.
> The problem is that I can only reasonably do this for boards
> I have, so there'd only be a few. But maybe that'd be enough.
> They would really only be meant as examples.
Start with qemu and work your way out? That's what I'm doing. (Any platform
that isn't emulated by qemu is almost by definition not very interesting...)
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Greg Ungerer @ 2008-06-11 5:47 UTC (permalink / raw)
To: Rob Landley
Cc: Sam Ravnborg, Leon Woestenberg, David Woodhouse, linux-embedded
In-Reply-To: <200806102235.09598.rob@landley.net>
Hi Rob,
Rob Landley wrote:
> On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote:
>>> (Maybe I _am_ the only person who still cares about
>>> building on a host without perl. If I wasn't, somebody else would have
>>> acked the patch...)
>> perl is pretty standard
>
> An implementation is not the same thing as a standard. If you mean "there is
> one implementation everybody uses, ala excel and Word, and even the Perl guys
> can't reproduce it from scratch as parrot showed", then you're using a
> different definition of the word "standard" than I am.
>
> Or do you mean it comes preinstalled on most modern systems, the way Windows
> does, and who could object to that?
>
> I know from experience that it's an _amazing_ pain to try to cross compile the
> sucker...
Ain't that the truth!
>> and I fail to see the benefits of avoiding it.
>> For embedded development I see even less benefits as I assume
>> any sane embedded development environment are based on a
>> cross-toolchain so you do the build on a high perfomance box.
>>
>> Building everything for my arm board on the arm board would be a disater
>> for example.
>
> I build everything for my arm board natively, on an arm system running under
> qemu, calling out to the cross compiler via distcc to accelerate the limited
> parts of the process that cross compiling doesn't actually break. To get to
> that point, I cross compile just enough to get a native development
> environment, and then I avoid cross compiling from then on because it's an
> enormous source of complexity and random breakage.
Random breakage?
> I did this because throwing hardware at the problem is cheaper than throwing
> engineering time at the problem, because Moore's Law is on my side, and
Are you sure about that?
How well does qemu do SMP? At all?
Modern x86 and friends are getting most new performance from
more cores. A cross compile today can take advantage of those
for the most part. Your emulated system probably can't.
An order of magnitude compile time (or more) for a native build
is quite a high price to pay :-)
Regards
Greg
> because I find native compiling (where possible) more straightforward and
> conceptually cleaner.
>
> Before 2.6.25, I had a complete self-bootstrapping environment with seven
> packages (busybox, uClibc, make, gcc, binutils, bash, and linux). Building
> glibc needs perl, if you're using a uClibc based system you may _never_ need
> Perl, and could go up through Linux From Scratch and beyond to a fairly
> powerful system.
>
> I can probably natively build perl under the mini-native environment running
> inside qemu, but it wasn't needed before and there really wasn't a good
> reason to add it. The task being performed and the build dependency brought
> in to perform it are hugely disproportionate.
>
> Rob
--
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: gerg@snapgear.com
Secure Computing Corporation PHONE: +61 7 3435 2888
825 Stanley St, FAX: +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Rob Landley @ 2008-06-11 5:34 UTC (permalink / raw)
To: Will Newton
Cc: David Woodhouse, Jamie Lokier, Wolfgang Denk, Sam Ravnborg,
Leon Woestenberg, linux-embedded
In-Reply-To: <87a5b0800806100647r178e54c0qd34cbf26f6ce24d@mail.gmail.com>
On Tuesday 10 June 2008 08:47:20 Will Newton wrote:
> On Tue, Jun 10, 2008 at 2:33 PM, David Woodhouse <dwmw2@infradead.org>
wrote:
> > On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote:
> >> On Tue, Jun 10, 2008 at 2:12 PM, Jamie Lokier <jamie@shareable.org>
wrote:
> >> > Wolfgang Denk wrote:
> >> >> Being unable to do this just because we now also would need a native
> >> >> Perl is indeed a PITA...
> >> >
> >> > You can run the Perl bit with "ssh remote perl", and still do the rest
> >> > of the compile natively. It's not pretty, but workable.
> >>
> >> I'm not convinced it matters at all. Self hosting on an embedded
> >> architecture is, as has been mentioned, pretty pointless.
> >>
> >> Using a kernel compile as a test isn't such a great idea. Stress tests
> >> of that kind are not particularly useful for pinning down bugs - so
> >> your kernel compile failed, what now? Far better to use LTP tests or
> >> similar that are designed to be reproduceable and tunable for your
> >> system. For example I don't think I'll ever be able to self host a
> >> kernel build on a board with only 32Mb of on-board RAM.
> >
> > Actually, cross-building on NFS does tend to find a _lot_ of issues
> > which crop up with board ports; especially PCI arbitration, DMA
> > coherency, cache and MMU issues. LTP often doesn't catch the same
> > problems.
>
> It may trigger a number of bugs, I don't disagree, but as a test it is
> a blunt instrument. It's likely to be hard to reproduce and have an
> inconsistent failure mode. If you're really serious about testing it's
> not the best solution. It's like using gcc instead of memtest86 to
> test your memory. Eventually it might go wrong but you won't be much
> the wiser about why, or have any way to trim your testcase down so you
> can run it on an in-circuit emulator or pass it to your silicon
> vendor.
>
> > I agree that it's not so easy on a board with 32Mb of RAM, since that's
> > only 4,000,000 bytes -- but 32MiB ought to be _perfectly_ sufficient :)
>
> I would be surprised if it was possible to compile Linux with gcc 4.2
> with 32MiB of total system memory.
Haven't tried, but I generally run emulated builds in 128 megs of ram (on 32
bit hosts), and I use this:
# This tells gcc to aggressively garbage collect its internal data
# structures. Without this, gcc triggers the OOM killer trying to rebuild
# itself in 128 megs of ram, which is the QEMU default size.
export CFLAGS="--param ggc-min-expand=0 --param ggc-min-heapsize=8192"
Don't do that on a 64 bit host or your build will slow to a crawl, of course.
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Paul Mundt @ 2008-06-11 5:28 UTC (permalink / raw)
To: Sam Ravnborg
Cc: Jamie Lokier, Rob Landley, Leon Woestenberg, David Woodhouse,
linux-embedded
In-Reply-To: <20080610105034.GB1900@uranus.ravnborg.org>
On Tue, Jun 10, 2008 at 12:50:34PM +0200, Sam Ravnborg wrote:
> >
> > When did this policy change, so that it's now acceptable to depend on
> > Perl, which is roughly equivalent as a tool dependency?
>
> We have perl as a mandatory part of the kernel build in several places
> for various architectures.
> And I do not recall anyone submitting a bug that they could not build
> a kernel due to the perl dependency.
> But I am obviously well aware of that we use it for the time stuff.
>
And plenty of places in scripts/ have dependencies on either perl or
python already (and have for some time). Both are pretty ubiquitous these
days, whether people like it or not. There's not much point in trying to
keep the build limping along for people who don't want to set up these
tools on their platforms, since they're going to lose out on half of the
functionality anyways (checkstack, bloat-o-meter, etc.).
Building natively is a good stress tester, and does find a lot of bugs.
For that same reason, if you can build the kernel, you can build python
and perl natively on your platform, too. Some of us have already been
doing this for ages and have no idea what the fuss is about.
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Rob Landley @ 2008-06-11 5:25 UTC (permalink / raw)
To: Wolfgang Denk
Cc: Will Newton, Jamie Lokier, Sam Ravnborg, Leon Woestenberg,
David Woodhouse, linux-embedded
In-Reply-To: <20080610134932.B27D62430C@gemini.denx.de>
On Tuesday 10 June 2008 08:49:32 Wolfgang Denk wrote:
> In message <87a5b0800806100625m5a6d20dao47b884bff663c24c@mail.gmail.com> you
wrote:
> > I'm not convinced it matters at all. Self hosting on an embedded
> > architecture is, as has been mentioned, pretty pointless.
>
> YMMV...
>
> > system. For example I don't think I'll ever be able to self host a
> > kernel build on a board with only 32Mb of on-board RAM.
>
> It depends. We've done this on board with as little as 16 MB RAM.
There's also qemu. You can native build under emulation.
In addition, if you have a cross compiler but don't want to spend all your
time lying to ./configure, preventing gcc from linking against the host's
zlib or grabbing stuff out of /usr/include that your target hasn't got, or
trying to figure out why on EARTH the perl build decided to use x86 signal
numbers when you built it for mips, you can build natively inside the
emulator but use distcc to call out to the cross compiler through the virtual
network. This uses the compiler for the heavy lifting of compilation
_and_nothing_else_. (make runs natively inside the emulator, ./configure
runs inside the emulator, headers are #included inside the emulator,
libraries are linked inside the emulator, anything that wants to look
at /proc or sysinfo does it natively inside the emulator...)
And the thing is, QEMU is running on fast cheap x86 hardware with buckets of
memory and disk space, and Moore's Law is doubling the power of it every 18
months (whereas it's making a lot of embedded stuff cheaper and have longer
battery life instead, at least until we get to fully disposable computers).
So yay using x86 as your build envionment, but building natively under
emulation is now an alternative to trying to make cross compiling scale.
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-11 5:17 UTC (permalink / raw)
To: Adrian Bunk; +Cc: Tim Bird, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080610183004.GG11685@cs181133002.pp.htv.fi>
On Tuesday 10 June 2008 13:30:04 Adrian Bunk wrote:
> On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > Adrian Bunk wrote:
> >...
> >
> > > You need both, and ideally constantly done by the same person against
> > > Linus' tree, -next and -mm.
> > >
> > > Where to get your minimal configs from at the start is just a small
> > > thing at the beginning - don't underestimate the required manual work
> > > that will have to be done each week.
> >
> > This is probably why I haven't signed up for this myself previously.
> > I'd be interested in finding out the rate at which defconfigs
> > bitrot in mainline. My experience is that usually a 'make oldconfig'
> > will produce something usable. But maybe that wouldn't be as
> > effective with a minconfig?
> >...
>
> Someone has to run the 'make oldconfig' for all configs...
Running "make oldconfig" isn't necessarily enough. If you can't build the
result, you don't really _know_ if it's going to work.
For example, in 2.6.23 new guard symbols showed up (CONFIG_BLK_DEV and
CONFIG_SCSI_LOWLEVEL), meaning if you had stuff under those selected but they
defaulted to off, everything under them would silently vanish. (I don't
remember what their defaults were, but I do remember it broke miniconfig.)
I need to go through and teach "make miniconfig" that when you set something
inside a menu, you set its menu symbol as well (all the way up to root if
necessary). That would allow the resulting config to strip down to fewer
symbols and not get broken by the addition of guard symbols between
versions...
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-11 5:09 UTC (permalink / raw)
To: Sam Ravnborg
Cc: Adrian Bunk, Tim Bird, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080610185123.GA15838@uranus.ravnborg.org>
On Tuesday 10 June 2008 13:51:23 Sam Ravnborg wrote:
> On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> > On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > > Adrian Bunk wrote:
> > >...
> > >
> > > > You need both, and ideally constantly done by the same person against
> > > > Linus' tree, -next and -mm.
> > > >
> > > > Where to get your minimal configs from at the start is just a small
> > > > thing at the beginning - don't underestimate the required manual work
> > > > that will have to be done each week.
> > >
> > > This is probably why I haven't signed up for this myself previously.
> > > I'd be interested in finding out the rate at which defconfigs
> > > bitrot in mainline. My experience is that usually a 'make oldconfig'
> > > will produce something usable. But maybe that wouldn't be as
> > > effective with a minconfig?
> > >...
> >
> > Someone has to run the 'make oldconfig' for all configs...
> >
> > And no, you cannot get that completely automated.
>
> When I get my kconfig patchset polished you will be able to do:
>
> make K=my_mini_config allnoconfig
So you're renaming KCONFIG_ALLNOCONFIG then?
> Thus selecting 'no' for all new symbols in an automated fashion.
> I know that in a few cases 'no' is the wrong answer but in the
> 99% of the cases 'no' is perfectly valid.
For a "make miniconfig", warnings should be errors. (Attempts to set unknown
symbols are an error with a miniconfig, the operation should exit with a
nonzero error code.) Also, all the stuff allnoconfig puts to stdout
shouldn't be there for miniconfig, only the stuff it writes to stderr right
now should show up.
I think I did more than that, but I'd have to look at my old patches...
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-11 3:48 UTC (permalink / raw)
To: Adrian Bunk; +Cc: Tim Bird, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <20080610083610.GC1987@cs181133002.pp.htv.fi>
On Tuesday 10 June 2008 03:36:10 Adrian Bunk wrote:
> On Mon, Jun 09, 2008 at 06:37:29PM -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> > >> At a minimum, it would be nice to have a few nice examples
> > >> of really, really small configs for things like qemus for different
> > >> architectures (just to give embedded developers who are working
> > >> on size a starting point).
> > >
> > > That's more or less what I'm trying to do with my Firmware Linux
> > > project: creating cross compilers and minimal native build environments
> > > for every qemu target.
> >
> > Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> It could help finding compile errors in some more "exotic" configurations
> early (but I'd question whether the Rob's current configs are really
> both minimal and typical for embedded usage - e.g the i686 one having
> both ext2 and ext3 enabled but no JFFS2; and enabling all IO schedulers
> in the kernel as well as ACPI is also not a typical embedded setup).
No argument there. I just offered them as a starting point for supporting
each qemu target.
I'm emulating a native build environment, meaning I need an emulated hard
drive with gibabytes of writable space; lots of embedded devices use
initramfs and nothing else. I'm using distcc so I need a working network
card and network stack, which lots of devices won't need. Also, some of them
(mips comes to mind) need to be stripped down some more.
(The ext2 plus ext3 thing is because ext2 isn't happy about writable mounts
where the emulator gets killed out from under it and then needs fsck, but
ext3 isn't really happy with small read-only mounts ala initrd. I keep
meaning to teach ext3 to work without a #*%&# journal, at least on read only
mounts, but it's about 150 entries down on my todo list...)
I keep meaning to refactor the configs into two files, so "just enough to boot
this with a serial console" is a separate file from things like filesystems
and network stack that are common to all of 'em. (Then concatenate the two
to get a miniconfig.) I'm not _quite_ convinced the extra build complexity's
worth it...
> But if you want to discover size change with minimal configs early you
> anyway have to both:
> - constantly keep your configs in shape so that they are both minimal
> for some set of hardware support and features and
I find miniconfig helps here.
> - investigate for any size changes what caused them
> (experience has shown that putting information on a webpage doesn't
> fix problems - even for compile errors).
>
> You need both, and ideally constantly done by the same person against
> Linus' tree, -next and -mm.
Speaking from experience, this is a huge #*%&# pain. (I'm trying to track
this sort of changes for less than a dozen qemu configs. There are hundreds
of defconfigs in the tree, most of which I can't boot test...)
However, qemu can be automated and scripted. (Especially when /dev/console
attaches to qemu's stdin and stdout. That's why I need each platform to know
how to shut _down_, and exit the emulator. Currently, powerpc -M prep
doesn't do this. :P)
> Where to get your minimal configs from at the start is just a small
> thing at the beginning - don't underestimate the required manual work
> that will have to be done each week.
Eh, I'd suggest every -pre release.
And starting with tracking the size regressions in just _one_ platform is
probably best. I'd suggest either x86 (32 bit, matches arm) or x86_64
(largest userbase at this point, even Via's finally switched over).
> > Does anyone else think this would be valuable? If not in mainline, it
> > would be nice to collect them somewhere, to compare what options
> > different developers decide turn on or off.
>
> You already have this when you look at e.g. the ARM defconfigs in the
> kernel
I've got a script running to convert all the 2.6.25 defconfigs into
miniconfigs, which I find makes 'em easier to read. I'll post the results
when it finishes...
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: [PATCH 0/1] Embedded Maintainer(s), linux-embedded@vger list
From: Rob Landley @ 2008-06-11 3:35 UTC (permalink / raw)
To: Sam Ravnborg; +Cc: Leon Woestenberg, David Woodhouse, linux-embedded
In-Reply-To: <20080610075432.GB776@uranus.ravnborg.org>
On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote:
> > (Maybe I _am_ the only person who still cares about
> > building on a host without perl. If I wasn't, somebody else would have
> > acked the patch...)
>
> perl is pretty standard
An implementation is not the same thing as a standard. If you mean "there is
one implementation everybody uses, ala excel and Word, and even the Perl guys
can't reproduce it from scratch as parrot showed", then you're using a
different definition of the word "standard" than I am.
Or do you mean it comes preinstalled on most modern systems, the way Windows
does, and who could object to that?
I know from experience that it's an _amazing_ pain to try to cross compile the
sucker...
> and I fail to see the benefits of avoiding it.
> For embedded development I see even less benefits as I assume
> any sane embedded development environment are based on a
> cross-toolchain so you do the build on a high perfomance box.
>
> Building everything for my arm board on the arm board would be a disater
> for example.
I build everything for my arm board natively, on an arm system running under
qemu, calling out to the cross compiler via distcc to accelerate the limited
parts of the process that cross compiling doesn't actually break. To get to
that point, I cross compile just enough to get a native development
environment, and then I avoid cross compiling from then on because it's an
enormous source of complexity and random breakage.
I did this because throwing hardware at the problem is cheaper than throwing
engineering time at the problem, because Moore's Law is on my side, and
because I find native compiling (where possible) more straightforward and
conceptually cleaner.
Before 2.6.25, I had a complete self-bootstrapping environment with seven
packages (busybox, uClibc, make, gcc, binutils, bash, and linux). Building
glibc needs perl, if you're using a uClibc based system you may _never_ need
Perl, and could go up through Linux From Scratch and beyond to a fairly
powerful system.
I can probably natively build perl under the mini-native environment running
inside qemu, but it wasn't needed before and there really wasn't a good
reason to add it. The task being performed and the build dependency brought
in to perform it are hugely disproportionate.
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: mainlining min-configs...
From: Rob Landley @ 2008-06-11 3:32 UTC (permalink / raw)
To: Tim Bird; +Cc: Adrian Bunk, linux-tiny, linux-embedded, linux kernel
In-Reply-To: <484DDAD9.2000306@am.sony.com>
On Monday 09 June 2008 20:37:29 Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> >
> > That's more or less what I'm trying to do with my Firmware Linux project:
> > creating cross compilers and minimal native build environments for every
> > qemu target.
>
> Any chance of getting your minimal configs from Firmware Linux mainlined?
There's _slightly_ more to it than that if you want to actually get a working
environment. (For example, I'm feeding ppc an extra patch and a boot rom,
both from Milton Miller. The config is useless without those. I can walk
you through the status and reasoning of each platform if you'd like...)
I have no objection to people taking the configs I worked out for my purposes
and using them for any purpose if they want to do so, but my idea
of "working" involves having a hard drive and a network connection (so I can
run builds under the emulator using distcc to call out to the cross
compiler). It's the minimal functionality _I_ need. I'm just offering it as
a starting point, because you specifically mentioned configs for qemu.
If you're looking to compare and contrast configurations, possibly a more
_useful_ thing would be to convert all the kernel's existing *_defconfig
files to *_miniconfig files so you could see what they all _are_.
Lemme take a stab at that, actually...
Rob
--
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Tim Bird @ 2008-06-11 0:25 UTC (permalink / raw)
To: Sam Ravnborg; +Cc: linux-embedded, linux kernel, David Woodhouse
In-Reply-To: <20080610212158.GB23855@uranus.ravnborg.org>
Sam Ravnborg wrote:
> I know this is unix style to be very short in usage - but then they
> have man pages.
> Could we add a bit more from the nice description above to usage?
Good idea. Might as well make it visible.
David Woodhouse wrote:
> It'd be really nice if it could give its output in the same form as
> the .config file itself -- so it'd look something like:
>
> # CONFIG_FOO is not set
> CONFIG_BAR=y
OK, a new version is below. Right now, when printing in "merge" style,
the code omits items that are absent in the new config. These should get set
correctly (removed) if you do a 'make oldconfig'. It would be trivial
to have them output as well, if you think that would be better.
I am experimentally trying to attach a git-style file mode
to this patch. I did this by hand, so it might not work.
(The index line is totally made up). Let me know if there are problems.
Diffconfig is a simple utility for comparing two .config files.
See usage in the script for more info.
Signed-off-by: Tim Bird <tim.bird@am.sony.com>
scripts/diffconfig | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 108 insertions(+)
create mode 100755 scripts/diffconfig
diff --git a/scripts/diffconfig b/scripts/diffconfig
new file mode 100755
index 0000000..aa6cfe1
--- /dev/null
+++ b/scripts/diffconfig
@@ -0,0 +1,108 @@
+#!/usr/bin/python
+#
+# diffconfig - a tool to compare .config files.
+#
+# originally written in 2006 by Matt Mackall
+# (at least, this was in his bloatwatch source code)
+# last worked on 2008 by Tim Bird
+#
+
+import sys, os
+
+merge_style = 0
+if "-m" in sys.argv:
+ merge_style = 1
+ sys.argv.remove("-m")
+
+if "-h" in sys.argv or len(sys.argv) < 3:
+ print "Usage: diffconfig [-h] [-m] <config1> <config2>\n"
+ print """Diffconfig is a simple utility for comparing two .config files.
+Using standard diff to compare .config files often includes
+extraneous and distracting information. This utility produces
+sorted output with only the changes in configuration values
+between the two files.
+
+Added and removed items are shown with a leading plus or minus,
+respectively. Changed items show the old and new values on a
+single line.
+
+If -m is specified, then output will be in "merge" style,
+which has the changed and new values in kernel config option format.
+
+Example usage:
+ $ diffconfig .config config-with-some-changes
+-EXT2_FS_XATTR n
+-EXT2_FS_XIP n
+ CRAMFS n -> y
+ EXT2_FS y -> n
+ LOG_BUF_SHIFT 14 -> 16
+ PRINTK_TIME n -> y
+"""
+ sys.exit(0)
+
+# returns a dictionary of name/value pairs for config items in the file
+def readconfig(config_file):
+ d = {}
+ for line in config_file:
+ line = line[:-1]
+ if line[:7] == "CONFIG_":
+ name, val = line[7:].split("=", 1)
+ d[name] = val
+ if line[-11:] == " is not set":
+ d[line[9:-11]] = "n"
+ return d
+
+a = readconfig(file(sys.argv[1]))
+b = readconfig(file(sys.argv[2]))
+
+def print_config(op, config, value, new_value):
+ if merge_style:
+ if new_value:
+ if new_value=="n":
+ print "# CONFIG_%s is not set" % config
+ else:
+ print "CONFIG_%s=%s" % (config, new_value)
+ else:
+ if op=="-":
+ print "-%s %s" % (config, value)
+ elif op=="+":
+ print "+%s %s" % (config, new_value)
+ else:
+ print " %s %s -> %s" % (config, value, new_value)
+
+# print items in a but not b (accumulate, sort and print)
+old = []
+for config in a:
+ if config not in b:
+ old.append(config)
+
+old.sort()
+
+for config in old:
+ print_config("-", config, a[config], None)
+ del a[config]
+
+# print items that changed (accumulate, sort, and print)
+changed = []
+for config in a:
+ if a[config] != b[config]:
+ changed.append(config)
+ else:
+ del b[config]
+
+changed.sort()
+
+for config in changed:
+ print_config("->", config, a[config], b[config])
+ del b[config]
+
+# now print items in b but not in a
+
+# the items from b that were in a (either the same or that changed) were removed
+# the only items left were not in a
+new = b.keys()
+
+new.sort()
+
+for config in new:
+ print_config("+", config, None, b[config])
^ permalink raw reply related
* Re: [PATCH] add diffconfig utility
From: David Woodhouse @ 2008-06-10 23:10 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: Tim Bird, linux-embedded, linux kernel
In-Reply-To: <20080610160202.3bb9d07f@infradead.org>
On Tue, 2008-06-10 at 16:02 -0700, Arjan van de Ven wrote:
> the person who wrote that tool also wrote a similar diff tool ;)
> but it's horrible perl I suspect.
Yeah, but I can't find it in CVS right now for some reason.
--
dwmw2
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Arjan van de Ven @ 2008-06-10 23:02 UTC (permalink / raw)
To: David Woodhouse; +Cc: Tim Bird, linux-embedded, linux kernel
In-Reply-To: <1213135624.2534.99.camel@shinybook.infradead.org>
On Tue, 10 Jun 2008 23:07:04 +0100
David Woodhouse <dwmw2@infradead.org> wrote:
> On Tue, 2008-06-10 at 12:41 -0700, Tim Bird wrote:
> > +# Added and removed items are shown with a leading plus or minus,
> > +# respectively. Changed items show the old and new values on a
> > +# single line.
>
> It'd be really nice if it could give its output in the same form as
> the .config file itself -- so it'd look something like:
>
> # CONFIG_FOO is not set
> CONFIG_BAR=y
>
> That can be used with this kind of tool too...
>
the person who wrote that tool also wrote a similar diff tool ;)
but it's horrible perl I suspect.
--
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings,
visit http://www.lesswatts.org
^ permalink raw reply
* Re: [PATCH] add diffconfig utility
From: Tim Bird @ 2008-06-10 22:33 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-embedded, linux kernel
In-Reply-To: <1213135624.2534.99.camel@shinybook.infradead.org>
David Woodhouse wrote:
> On Tue, 2008-06-10 at 12:41 -0700, Tim Bird wrote:
>> +# Added and removed items are shown with a leading plus or minus,
>> +# respectively. Changed items show the old and new values on a
>> +# single line.
>
> It'd be really nice if it could give its output in the same form as
> the .config file itself -- so it'd look something like:
>
> # CONFIG_FOO is not set
> CONFIG_BAR=y
Would an option for that style of output be OK?
The tool currently produces something similar
to a diff (but with changed lines compressed to
a single line for conciseness). It should be easy
to produce something that can be used with another
tool to transform one config into another.
It's unclear what to do with variables that are
supposed to be removed from the config (as opposed
to be set to "CONFIG_FOO is not set" Does merge.pl
handle this, or is this left to something like
'make oldconfig'?
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox