Cross Compiler and loads of issues

Cross Compiler and loads of issues

[Shaz]

Hi,

I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
felt like asking that is there one good way to get a cross compiler
work. I tried buildroot, scratchbox and even openMoko with
openEmbedded but all of them had lots of issues and don't know which
will be the best alternative.

I also went through the material provided freely by Free Electron but
still I am not successful to build a custom kernel. Next I am trying
MontaVista's kit. I just wish I don't get lost.

Anyways, I liked the idea of Qemu based cross compiler. Is it possible
for the inexperienced to get it working and emulate the exact model
and devices.

--

Shaz

Re: Cross Compiler and loads of issues

[Wolfgang Denk]

In message <7b740b700806121052n2f98dfa4hc96ebfc1be5b6bbf@mail.gmail.com> you wrote:
> 
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.
> 
> I also went through the material provided freely by Free Electron but
> still I am not successful to build a custom kernel. Next I am trying
> MontaVista's kit. I just wish I don't get lost.

For completeness, there is also the (free) ELDK. Some people are using it.

[And yes, I'm obviously biased :-) ]

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
A list is only as strong as its weakest link.            -- Don Knuth

Re: Cross Compiler and loads of issues

[Shaz]

On Thu, Jun 12, 2008 at 11:19 PM, Wolfgang Denk <wd@denx.de> wrote:
> In message <7b740b700806121052n2f98dfa4hc96ebfc1be5b6bbf@mail.gmail.com> you wrote:
>>
>> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
>> felt like asking that is there one good way to get a cross compiler
>> work. I tried buildroot, scratchbox and even openMoko with
>> openEmbedded but all of them had lots of issues and don't know which
>> will be the best alternative.
>>
>> I also went through the material provided freely by Free Electron but
>> still I am not successful to build a custom kernel. Next I am trying
>> MontaVista's kit. I just wish I don't get lost.
>
> For completeness, there is also the (free) ELDK. Some people are using it.
>

I got a toolchain giving an output. Going to test some stuff with
Qemu. I hope it works. ELDK seems to be a smooth tool. Now got to get
it working with my kernel, eclipse and target board :)

> [And yes, I'm obviously biased :-) ]
>
> Best regards,
>
> Wolfgang Denk
>
> --
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
> A list is only as strong as its weakest link.            -- Don Knuth
>



-- 
Shaz

Re: Cross Compiler and loads of issues

[Matthias Kaehlcke]

El Thu, Jun 12, 2008 at 10:52:44PM +0500 Shaz ha dit:

> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

crosstool-ng looks like a promising option:
http://ymorin.is-a-geek.org/dokuwiki/projects/crosstool
 
-- 
Matthias Kaehlcke
Embedded Linux Engineer
Barcelona

     Nationalism is an infantile disease. It is the measles of mankind
                         (Albert Einstein)
                                                                 .''`.
    using free software / Debian GNU/Linux | http://debian.org  : :'  :
                                                                `. `'`
gpg --keyserver pgp.mit.edu --recv-keys 47D8E5D4                  `-

Re: Cross Compiler and loads of issues

[Bill Traynor]

> Hi,
>
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

There is no "one good way".  I've had decent success building Dan Kegel's
"crosstool" in the past: http://www.kegel.com/crosstool/
>
> I also went through the material provided freely by Free Electron but
> still I am not successful to build a custom kernel. Next I am trying
> MontaVista's kit. I just wish I don't get lost.

I'd continue the search for prebuilt toolchains.  CodeSourcery has a Lite
version of it's ARM, Coldfire, MIPS, and Power toolchains.

>
> Anyways, I liked the idea of Qemu based cross compiler. Is it possible
> for the inexperienced to get it working and emulate the exact model
> and devices.
>
> --
>
> Shaz
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Cross Compiler and loads of issues

[Enrico Weigelt]

* Bill Traynor <wmat@naoi.ca> schrieb:

> There is no "one good way".  I've had decent success building Dan Kegel's
> "crosstool" in the past: http://www.kegel.com/crosstool/

I'd also like to mention Yann's crosstool-ng :)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: Cross Compiler and loads of issues

[Shaz]

On Thu, Jun 12, 2008 at 11:49 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Bill Traynor <wmat@naoi.ca> schrieb:
>
>> There is no "one good way".  I've had decent success building Dan Kegel's
>> "crosstool" in the past: http://www.kegel.com/crosstool/
>
> I'd also like to mention Yann's crosstool-ng :)
I guess I have to reconsider cross-tool because earlier I visited this
http://www.kegel.com/crosstool/crosstool-0.43/buildlogs/ and I want to
get the latest kernel up and running on OSK OMAP5912, arm9. The board
is in process of shipment but I started out with a qemu
(arm-generic/versatilepb) based test run.

I am in the middle of a confused experience :-)
>
>
> cu
> --
> ---------------------------------------------------------------------
>  Enrico Weigelt    ==   metux IT service - http://www.metux.de/
> ---------------------------------------------------------------------
>  Please visit the OpenSource QM Taskforce:
>        http://wiki.metux.de/public/OpenSource_QM_Taskforce
>  Patches / Fixes for a lot dozens of packages in dozens of versions:
>        http://patches.metux.de/
> ---------------------------------------------------------------------
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Shaz

Security Engineering Research Group,
Institute of Management Sciences,
Peshawar, Pakistan

Group: http://serg.imsciences.edu.pk

Email: shazalive@gmail.com
cell: +92 300 5944647

Re: Cross Compiler and loads of issues

[George G. Davis]

On Fri, Jun 13, 2008 at 12:02:06AM +0500, Shaz wrote:
> On Thu, Jun 12, 2008 at 11:49 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> > * Bill Traynor <wmat@naoi.ca> schrieb:
> >
> >> There is no "one good way".  I've had decent success building Dan Kegel's
> >> "crosstool" in the past: http://www.kegel.com/crosstool/
> >
> > I'd also like to mention Yann's crosstool-ng :)
> I guess I have to reconsider cross-tool because earlier I visited this
> http://www.kegel.com/crosstool/crosstool-0.43/buildlogs/ and I want to
> get the latest kernel up and running on OSK OMAP5912, arm9. The board
> is in process of shipment but I started out with a qemu
> (arm-generic/versatilepb) based test run.
> 
> I am in the middle of a confused experience :-)

I've used this to build cross compilers (which worked ok for me : ):

http://code.google.com/p/jicknan/source/browse/trunk/bin/toolchain-eglibc.sh

As usual, YMMV, good luck, HTH!

--
Regards,
George

Re: Cross Compiler and loads of issues

[Bill Traynor]

>> Hi,
>>
>> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
>> felt like asking that is there one good way to get a cross compiler
>> work. I tried buildroot, scratchbox and even openMoko with
>> openEmbedded but all of them had lots of issues and don't know which
>> will be the best alternative.
>
> There is no "one good way".  I've had decent success building Dan Kegel's
> "crosstool" in the past: http://www.kegel.com/crosstool/
>>
>> I also went through the material provided freely by Free Electron but
>> still I am not successful to build a custom kernel. Next I am trying
>> MontaVista's kit. I just wish I don't get lost.
>
> I'd continue the search for prebuilt toolchains.  CodeSourcery has a Lite
> version of it's ARM, Coldfire, MIPS, and Power toolchains.

I should have pointed this out before, but there is a useful Toolchains
wiki page here:  http://elinux.org/Toolchains

Feel free to contribute findings to that page.

>
>>
>> Anyways, I liked the idea of Qemu based cross compiler. Is it possible
>> for the inexperienced to get it working and emulate the exact model
>> and devices.
>>
>> --
>>
>> Shaz
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-embedded"
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Cross Compiler and loads of issues

[Bill Gatliff]

Shaz wrote:
> Hi,
> 
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

I like running emdebian's toolchains on a Debian box, myself.


b.g.
-- 
Bill Gatliff
bgat@billgatliff.com

Re: Cross Compiler and loads of issues

[Glenn Henshaw]

On 12-Jun-08, at 1:52 PM, Shaz wrote:

> Hi,
>
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

   I've built compiler chains for clients using buildroot.

 From my notes:
  * download buildroot and expand it in your home directory
  * make menuconfig - select kernel, compiler, and uclibc versions.  
(2.4.27, 3.4.3, and 0.9.28 respectively), set target to final  
destination ,i.e., /opt/armeb as you can't move the compiler once built
  * make - this will download compiler, etc and build it. It will most  
likely fail the first time.
  * patches to apply (http://bugs.uclibc.org/view.php?id=51 - fix LFS  
requirements)
  * make - again to finish

    ... Glenn


-- 
Glenn Henshaw                     Logical Outcome Ltd.
e: thraxisp@logicaloutcome.ca     w: www.logicaloutcome.ca

Re: Cross Compiler and loads of issues

[Robert Schwebel]

On Thu, Jun 12, 2008 at 10:52:44PM +0500, Shaz wrote:
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

There's also OSELAS.Toolchain:
http://www.pengutronix.de/oselas/toolchain/index_en.html

We try to be as close to mainline gcc/binutils as possible, and if there
are issues, please report or send patches :-)

> Anyways, I liked the idea of Qemu based cross compiler. Is it possible
> for the inexperienced to get it working and emulate the exact model
> and devices.

No.

rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Re: Cross Compiler and loads of issues

[Wang, Baojun]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

在 2008-06-13五的 06:34 +0200，Robert Schwebel写道：
> On Thu, Jun 12, 2008 at 10:52:44PM +0500, Shaz wrote:
> > I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> > felt like asking that is there one good way to get a cross compiler
> > work. I tried buildroot, scratchbox and even openMoko with
> > openEmbedded but all of them had lots of issues and don't know which
> > will be the best alternative.
If you only need to use the cross toolchain, I think ELDK is a good choice. other alternatives would be gentoo cross toolchain (using crossdev ie crossdev -t powerpc64-unknownlinux-gnu) and emdebian/slind.

Gentoo cross toolchain is my favorite, it very simple to (e)merge (just
crossdev -t arch-vendor-platform-libc) and the toolchain is managed by
gentoo package manager (portage) so all packages in the toolchain is
upgradable. I've installed about 12 different (gentoo) cross toolchain
on one machine for cross compiling.

Personally I don't use emdebian/slind but still I think they're also a
good choice.

> There's also OSELAS.Toolchain:
> http://www.pengutronix.de/oselas/toolchain/index_en.html
> 
> We try to be as close to mainline gcc/binutils as possible, and if there
> are issues, please report or send patches :-)
> 
> > Anyways, I liked the idea of Qemu based cross compiler. Is it possible
> > for the inexperienced to get it working and emulate the exact model
> > and devices.
> 
> No.
> 
> rsc
-- 
Wang, Baojun                                                Lanzhou University
Distributed & Embedded System Lab                      http://dslab.lzu.edu.cn
School of Information Science and Engeneering          wangbj@dslab.lzu.edu.cn
Tianshui South Road 222. Lanzhou 730000                             .P.R.China
Tel: +86-931-8912025                                      Fax: +86-931-8912022


[-- Attachment #2: 这是信件的数字签名部分 --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Cross Compiler and loads of issues

[Wookey]

On 2008-06-12 22:52 +0500, Shaz wrote:
> Hi,
> 
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

'issues' are generally par for the course. It's a difficult problem.
I've generally found buildroot the least-painful of the above, largely
because it targets a relatively small section of the problem-space. 

For completeness I should also mention that Embedded Debian provides
yet another option. I will not claim that it will have fewer issues
than the above. http://www.emdebian.org/emdebian/intro.html

I'm not sure if you are really asking about toolchains or build-systems?
The former is fairly reliably solved. For a debian-based box just
install pre-built cross toolchians from emdebian:
http://www.emdebian.org/tools/crosstools.html
(from i386, amd64, powerpc; to: nearly all debian-supported
architectures, gcc3.3, 3.4, 4.1, 4.2, 4.3)
http://www.emdebian.org/toolchains/search.php?section=devel gives more
details.

emdebian-tools also provides the debian equivalent of crosstool
('emchain' to build your own version of current current toolchain,
should a suitable pre-built one not exist.

For non-debian boxes other people seem to have listed the options so I
won't repeat (and it's not my area of expertise).

Wookey (Emdebian 'Chief bullshitter' - bias alert. The toolchains are
great though, really.)
-- 
Principal hats:  Balloonz - Toby Churchill - Aleph One - Debian
http://wookware.org/

Re: Cross Compiler and loads of issues

[Shaz]

It's nice to see we have so many options and related people and pros
to it are available around.

IMO there should be some sort of effort to standardize the tool-chains
and build environments coherently with the kernel. I think its a prime
time to work around all the possibilities and standardize so that a
collaborative effort similar to Linux kernel can be set ahead. I think
thats the objective of this list too. Any plans yet sorted out?

On Fri, Jun 13, 2008 at 2:24 PM, Wookey <wookey@wookware.org> wrote:
>
> On 2008-06-12 22:52 +0500, Shaz wrote:
> > Hi,
> >
> > I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> > felt like asking that is there one good way to get a cross compiler
> > work. I tried buildroot, scratchbox and even openMoko with
> > openEmbedded but all of them had lots of issues and don't know which
> > will be the best alternative.
>
> 'issues' are generally par for the course. It's a difficult problem.
> I've generally found buildroot the least-painful of the above, largely
> because it targets a relatively small section of the problem-space.
>
> For completeness I should also mention that Embedded Debian provides
> yet another option. I will not claim that it will have fewer issues
> than the above. http://www.emdebian.org/emdebian/intro.html
>
> I'm not sure if you are really asking about toolchains or build-systems?
> The former is fairly reliably solved. For a debian-based box just
> install pre-built cross toolchians from emdebian:
> http://www.emdebian.org/tools/crosstools.html
> (from i386, amd64, powerpc; to: nearly all debian-supported
> architectures, gcc3.3, 3.4, 4.1, 4.2, 4.3)
> http://www.emdebian.org/toolchains/search.php?section=devel gives more
> details.
>
> emdebian-tools also provides the debian equivalent of crosstool
> ('emchain' to build your own version of current current toolchain,
> should a suitable pre-built one not exist.
>
> For non-debian boxes other people seem to have listed the options so I
> won't repeat (and it's not my area of expertise).
>
> Wookey (Emdebian 'Chief bullshitter' - bias alert. The toolchains are
> great though, really.)
> --
> Principal hats:  Balloonz - Toby Churchill - Aleph One - Debian
> http://wookware.org/



--
Shaz

Re: Cross Compiler and loads of issues

[Bill Traynor]

> It's nice to see we have so many options and related people and pros
> to it are available around.
>
> IMO there should be some sort of effort to standardize the tool-chains
> and build environments coherently with the kernel. I think its a prime
> time to work around all the possibilities and standardize so that a
> collaborative effort similar to Linux kernel can be set ahead. I think
> thats the objective of this list too. Any plans yet sorted out?

I don't understand what you mean by "standardize the tool-chains and build
environments", can you provide specifics?  Similarly, what does "work
around all the possibilities and standardize" mean?

Maybe I'm being dense, but what's specifically wrong with the current
toolchain universe?
>
> On Fri, Jun 13, 2008 at 2:24 PM, Wookey <wookey@wookware.org> wrote:
>>
>> On 2008-06-12 22:52 +0500, Shaz wrote:
>> > Hi,
>> >
>> > I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
>> > felt like asking that is there one good way to get a cross compiler
>> > work. I tried buildroot, scratchbox and even openMoko with
>> > openEmbedded but all of them had lots of issues and don't know which
>> > will be the best alternative.
>>
>> 'issues' are generally par for the course. It's a difficult problem.
>> I've generally found buildroot the least-painful of the above, largely
>> because it targets a relatively small section of the problem-space.
>>
>> For completeness I should also mention that Embedded Debian provides
>> yet another option. I will not claim that it will have fewer issues
>> than the above. http://www.emdebian.org/emdebian/intro.html
>>
>> I'm not sure if you are really asking about toolchains or build-systems?
>> The former is fairly reliably solved. For a debian-based box just
>> install pre-built cross toolchians from emdebian:
>> http://www.emdebian.org/tools/crosstools.html
>> (from i386, amd64, powerpc; to: nearly all debian-supported
>> architectures, gcc3.3, 3.4, 4.1, 4.2, 4.3)
>> http://www.emdebian.org/toolchains/search.php?section=devel gives more
>> details.
>>
>> emdebian-tools also provides the debian equivalent of crosstool
>> ('emchain' to build your own version of current current toolchain,
>> should a suitable pre-built one not exist.
>>
>> For non-debian boxes other people seem to have listed the options so I
>> won't repeat (and it's not my area of expertise).
>>
>> Wookey (Emdebian 'Chief bullshitter' - bias alert. The toolchains are
>> great though, really.)
>> --
>> Principal hats:  Balloonz - Toby Churchill - Aleph One - Debian
>> http://wookware.org/
>
>
>
> --
> Shaz
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Cross Compiler and loads of issues

[Jamie Lokier]

Bill Traynor wrote:
> Maybe I'm being dense, but what's specifically wrong with the current
> toolchain universe?

Back in ye olde days, you could download GCC and Binutils from
gnu.org, configure for whatever is your architecture, and most times
it just worked.

For some reason, that stopped a while ago, and you had to go to
different places to get working basic tools.  And often, the place to
go wasn't clear.  Different people advertised their "ARM toolchain",
"m68k toolchain" etc.  and they were slightly different sets of
patches on top of mainline tools.  Central authorities you might
believe in existed, but they were always a few patches behind what you
actually needed.

When I last needed a toolchain, Google led to confusing results, and I
had to try more than one.  I still use mutiple GCC versions (from
different people) to compile different programs for the same
architecture: each one fails some things in a different way, including
run-time failures in the precompiled toolchains.

Just Google for "uclinux toolchain" and the top hits lead to very old
releases, with bugs that have long been fixed elsewhere.  "uclinux arm
toolchain" is no better.

Perhaps current versions (e.g. from Codesourcery?) are more dependable
for embedded architectures, but I don't have the time to thoroughly
test them, and my last experience warns me to be careful.

It seems people release tools, release patches, publish on an obscure
web page, then forget about the page.  More authoritative-sounding
uclinux web pages tend to be out of date.  Google isn't finding good
current authorities in this area, which suggests the space is rather
fragmented with people pulling in different directions and not working
together enough to create stable, common places for these things.

Contrast with kernel.org: everyone knows where to get a good working
Linux kernel for the mainstream architectures, and the quality work
tends to be quite good at reaching mainline there nowadays.

-- Jamie

Re: Cross Compiler and loads of issues

[Enrico Weigelt]

* Jamie Lokier <jamie@shareable.org> schrieb:

> For some reason, that stopped a while ago, and you had to go to
> different places to get working basic tools.  And often, the place to
> go wasn't clear.  Different people advertised their "ARM toolchain",
> "m68k toolchain" etc.  and they were slightly different sets of
> patches on top of mainline tools.  Central authorities you might
> believe in existed, but they were always a few patches behind what you
> actually needed.

That's because many embedded build their toolchains completely on
their own, instead of just using an generic toolkit like ct.

> Contrast with kernel.org: everyone knows where to get a good working
> Linux kernel for the mainstream architectures, and the quality work
> tends to be quite good at reaching mainline there nowadays.

ACK. But you perhaps remember the discussions on LKML where some
folks wanted to stop this and leave all the QM works to individual
distros. I'm glad this plan was dropped.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: Cross Compiler and loads of issues

[Jamie Lokier]

Enrico Weigelt wrote:
> > Contrast with kernel.org: everyone knows where to get a good working
> > Linux kernel for the mainstream architectures, and the quality work
> > tends to be quite good at reaching mainline there nowadays.
> 
> ACK. But you perhaps remember the discussions on LKML where some
> folks wanted to stop this and leave all the QM works to individual
> distros. I'm glad this plan was dropped.

I'm glad too.  I've had to do the reverse: cherry-pick through 2000
patches from distro kernel source packages, to find the good ones for
my kernel - bug fixes, driver fixes.  It took a long time, and I had
to give up before finishing, it was simply too much work.

That was 2.4, back when distros did a lot of their own patches, kept
outside the mainline kernel.  Thankfully, the 2.6 process is much better.

-- Jamie

Re: Cross Compiler and loads of issues

[Bill Traynor]

> Bill Traynor wrote:
>> Maybe I'm being dense, but what's specifically wrong with the current
>> toolchain universe?
>
> Back in ye olde days, you could download GCC and Binutils from
> gnu.org, configure for whatever is your architecture, and most times
> it just worked.

Yes, the difficulty is in the "configure" step.  But the GNU Toolchain is
a complex beast but the benefit is it flexibility and numerous supported
architectures.  Ye olde days perhaps was less complex, but also less
flexible and supporting less archs.

>
> For some reason, that stopped a while ago, and you had to go to
> different places to get working basic tools.  And often, the place to
> go wasn't clear.  Different people advertised their "ARM toolchain",
> "m68k toolchain" etc.  and they were slightly different sets of
> patches on top of mainline tools.  Central authorities you might
> believe in existed, but they were always a few patches behind what you
> actually needed.

I disagree with respect to ARM.  GNU toolchains for ARM have for the last
four or five years been relatively up to date for ARM hardware.  This is a
direct result of ARM paying for this support to be added.

>
> When I last needed a toolchain, Google led to confusing results, and I
> had to try more than one.  I still use mutiple GCC versions (from
> different people) to compile different programs for the same
> architecture: each one fails some things in a different way, including
> run-time failures in the precompiled toolchains.

You didn't have to try more than one, you chose to try more than one.  You
could have just as easily chosen to use only the current mainline GNU
toolchain and worked through your issues with the community, thereby
allowing mainline to benefit from your fixes, and you to benefit by
getting to use the most current toolchain.

>
> Just Google for "uclinux toolchain" and the top hits lead to very old
> releases, with bugs that have long been fixed elsewhere.  "uclinux arm
> toolchain" is no better.

The "fixed elsewhere" is the problem.  If everyone used the most current
release and worked through issues with the community, this problem would
go away.

>
> Perhaps current versions (e.g. from Codesourcery?) are more dependable
> for embedded architectures, but I don't have the time to thoroughly
> test them, and my last experience warns me to be careful.

I know from experience, having worked for CodeSourcery that their
toolchains are exhaustively tested.  And any problems reported to their
public mailing lists are fixed, if legitimate.  These fixes are typically
submitted upstream to the FSF very quickly when the bugs exist in mainline
as well.

>
> It seems people release tools, release patches, publish on an obscure
> web page, then forget about the page.  More authoritative-sounding
> uclinux web pages tend to be out of date.  Google isn't finding good
> current authorities in this area, which suggests the space is rather
> fragmented with people pulling in different directions and not working
> together enough to create stable, common places for these things.

But isn't the FSF repositories for the GNU Tools, and the uClinux projects
repositories the stable, common place for these things?

>
> Contrast with kernel.org: everyone knows where to get a good working
> Linux kernel for the mainstream architectures, and the quality work
> tends to be quite good at reaching mainline there nowadays.

IMHO, the same is true for the GNU toolchain.


>
> -- Jamie
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Cross Compiler and loads of issues

[Enrico Weigelt]

* Bill Traynor <wmat@naoi.ca> schrieb:

> The "fixed elsewhere" is the problem.  If everyone used the most current
> release and worked through issues with the community, this problem would
> go away.

Yep, and here we're again at the point that opensource/community 
development and just use open code to build something own are 
different things ;-o


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: Cross Compiler and loads of issues

[Jamie Lokier]

Bill Traynor wrote:
> > For some reason, that stopped a while ago, and you had to go to
> > different places to get working basic tools.  And often, the place to
> > go wasn't clear.  Different people advertised their "ARM toolchain",
> > "m68k toolchain" etc.  and they were slightly different sets of
> > patches on top of mainline tools.  Central authorities you might
> > believe in existed, but they were always a few patches behind what you
> > actually needed.
> 
> I disagree with respect to ARM.  GNU toolchains for ARM have for the last
> four or five years been relatively up to date for ARM hardware.  This is a
> direct result of ARM paying for this support to be added.

I believe you, but they were difficult to _find_.  A couple of years
ago I looked for up to date ARM GNU tools.  The first few pages from
Google lead to a small number of places where you could download
toolchains at least a year out of date, possibly more, and none to get
anything current in a straightforward format (i.e. not an entire dev
kit with IDEs etc.).

That situation has improved greatly since then.

But, to be honest, ARM-nommu toolchain support is still second class.

There's no shared library / dynamic loader support for ARM-nommu, and
no apparent plans to implement it.  (I'd help but I don't have time to
do it all).  NPTL threads are coming along, but not yet ready; it's
been quite slow.  C++ sometimes crashes at the linking stage.

> > When I last needed a toolchain, Google led to confusing results, and I
> > had to try more than one.  I still use mutiple GCC versions (from
> > different people) to compile different programs for the same
> > architecture: each one fails some things in a different way, including
> > run-time failures in the precompiled toolchains.
> 
> You didn't have to try more than one, you chose to try more than one.

I found significant problems with the first.  Fixing obscure looking
register allocation crash bugs in the compiler was more than I thought
I had time for - looking for another was less work than that,
therefore I rationally chose it.

I figured that bug was fixed in a newer version, and it was.  Then I
found run-time problems with threads (crashing and impossible to trap
in GDB - probably distributed libc mismatched my kernel - wtf?), and
incompatibilities with binary libraries I had to link with.  (That's
why I use two different toolchain versions in my project now).

Eventually, for some applications, what worked was a combination:
tools from one place but headers and libraries from the other.

To solve this properly, requires that I delve into the toolchain code,
or find _another_ newer one which might fix these problems.

It's a substantial time sink to do any of these things.

> You could have just as easily chosen to use only the current
> mainline GNU toolchain and worked through your issues with the
> community, thereby allowing mainline to benefit from your fixes, and
> you to benefit by getting to use the most current toolchain.

I agree that is good.  However, my issues were rather specific to my
application setup and third party commercial dependencies (typical of
some embedded projects), and depended on older tools and kernels, so I
wouldn't expect the community to have much interest.  The lack of
interest in 2.4 kernels is pretty explicit (understandably - I don't
like using it either), and for GCC 2.old/3.old I'm assuming it.

The lack of backed-by-action interest in making ARM-nommu tools
support current GCC features like C++ properly, shared libraries, and
modern threads, adds to the sense that, while it's good to be in touch
with the community, there isn't a lot of readiness to work on obscure
things which don't affect a lot of people, and on hardware which isn't
the future.

Eventually I might forward port enough to follow mainline kernels and
tools.  But that's a substantial time sink too - don't underestimate
it.  I'm told that mainline current 2.6 doesn't build let alone work
on ARM-nommu (yet); that does not make forward-porting including local
drivers and architecture quirks sound like it will be quick.  If slow,
it's not worth the effort to me alone.

> > Just Google for "uclinux toolchain" and the top hits lead to very old
> > releases, with bugs that have long been fixed elsewhere.  "uclinux arm
> > toolchain" is no better.
> 
> The "fixed elsewhere" is the problem.  If everyone used the most current
> release and worked through issues with the community, this problem would
> go away.

I agree.  I would like to do that, and with a thriving, interested
community I will.  I'm hoping linux-embedded@vger.kernel.org might
help catalyse it - perhaps by creating the perception of it to people
like me.

It only really works if lots of people do that together, and it seems
to be particularly _not_ the culture in some segments of the embedded
development community.

I base this on Google leading to lots of pages with old announcements
of compilers, cross-compilation scripts and so on.  Why aren't those
pages links to a few standard "good" places which are actively kept up
to date?  It appears to be (or have been) a fragmented community - and
so who to work with, that's a question.

> > Perhaps current versions (e.g. from Codesourcery?) are more dependable
> > for embedded architectures, but I don't have the time to thoroughly
> > test them, and my last experience warns me to be careful.
> 
> I know from experience, having worked for CodeSourcery that their
> toolchains are exhaustively tested.  And any problems reported to their
> public mailing lists are fixed, if legitimate.  These fixes are typically
> submitted upstream to the FSF very quickly when the bugs exist in mainline
> as well.

I know from experience that "bugs" are fixed, but feature requests
like reliable C++, shared libraries and NPTL threads are long in
coming to some architectures, if they ever will be before those
architectures are no longer manufactured.

> > It seems people release tools, release patches, publish on an obscure
> > web page, then forget about the page.  More authoritative-sounding
> > uclinux web pages tend to be out of date.  Google isn't finding good
> > current authorities in this area, which suggests the space is rather
> > fragmented with people pulling in different directions and not working
> > together enough to create stable, common places for these things.
> 
> But isn't the FSF repositories for the GNU Tools, and the uClinux projects
> repositories the stable, common place for these things?

You can't even build an ARM-nommu executable with GNU tools from FSF!

It needs an extra tool.  I'm not even sure which version of that is
the recommended one - the C one or the shell script both have issues
outstanding.  By the way, neither work with some C++ programs on
ARM-nommu: they crash with messages about unfixable relocations.  More
often with GCC 4 than GCC 3.  See what I mean about things not being
in great shape?

> > Contrast with kernel.org: everyone knows where to get a good working
> > Linux kernel for the mainstream architectures, and the quality work
> > tends to be quite good at reaching mainline there nowadays.
> 
> IMHO, the same is true for the GNU tool.

If that's true for ARM-nommu support, it has improved dramatically.  I
will try again, in time, but prior experience tells me not to spend
too long on it.  It's much better use of time to find some patched
combination that is working well for someone else.  But who to
ask/trust?  Even the pre-built toolchains don't always work right.
(See my earlier mention of undebuggable thread problems in executables
built with a googd-distributed.)

-- Jamie

Re: Cross Compiler and loads of issues

[Greg Ungerer]

Jamie Lokier wrote:
> Bill Traynor wrote:
>>> For some reason, that stopped a while ago, and you had to go to
>>> different places to get working basic tools.  And often, the place to
>>> go wasn't clear.  Different people advertised their "ARM toolchain",
>>> "m68k toolchain" etc.  and they were slightly different sets of
>>> patches on top of mainline tools.  Central authorities you might
>>> believe in existed, but they were always a few patches behind what you
>>> actually needed.
>> I disagree with respect to ARM.  GNU toolchains for ARM have for the last
>> four or five years been relatively up to date for ARM hardware.  This is a
>> direct result of ARM paying for this support to be added.
> 
> I believe you, but they were difficult to _find_.  A couple of years
> ago I looked for up to date ARM GNU tools.  The first few pages from
> Google lead to a small number of places where you could download
> toolchains at least a year out of date, possibly more, and none to get
> anything current in a straightforward format (i.e. not an entire dev
> kit with IDEs etc.).
> 
> That situation has improved greatly since then.
> 
> But, to be honest, ARM-nommu toolchain support is still second class.
> 
> There's no shared library / dynamic loader support for ARM-nommu, and
> no apparent plans to implement it.  (I'd help but I don't have time to
> do it all).  NPTL threads are coming along, but not yet ready; it's
> been quite slow.  C++ sometimes crashes at the linking stage.

Nobody is willing to put in the new effort here required to
make this work. So the "no plans" really means no person power
to make it happen.


>>> When I last needed a toolchain, Google led to confusing results, and I
>>> had to try more than one.  I still use mutiple GCC versions (from
>>> different people) to compile different programs for the same
>>> architecture: each one fails some things in a different way, including
>>> run-time failures in the precompiled toolchains.
>> You didn't have to try more than one, you chose to try more than one.
> 
> I found significant problems with the first.  Fixing obscure looking
> register allocation crash bugs in the compiler was more than I thought
> I had time for - looking for another was less work than that,
> therefore I rationally chose it.
> 
> I figured that bug was fixed in a newer version, and it was.  Then I
> found run-time problems with threads (crashing and impossible to trap
> in GDB - probably distributed libc mismatched my kernel - wtf?), and
> incompatibilities with binary libraries I had to link with.  (That's
> why I use two different toolchain versions in my project now).
> 
> Eventually, for some applications, what worked was a combination:
> tools from one place but headers and libraries from the other.
> 
> To solve this properly, requires that I delve into the toolchain code,
> or find _another_ newer one which might fix these problems.
> 
> It's a substantial time sink to do any of these things.
> 
>> You could have just as easily chosen to use only the current
>> mainline GNU toolchain and worked through your issues with the
>> community, thereby allowing mainline to benefit from your fixes, and
>> you to benefit by getting to use the most current toolchain.
> 
> I agree that is good.  However, my issues were rather specific to my
> application setup and third party commercial dependencies (typical of
> some embedded projects), and depended on older tools and kernels, so I
> wouldn't expect the community to have much interest.  The lack of
> interest in 2.4 kernels is pretty explicit (understandably - I don't
> like using it either), and for GCC 2.old/3.old I'm assuming it.
> 
> The lack of backed-by-action interest in making ARM-nommu tools
> support current GCC features like C++ properly, shared libraries, and
> modern threads, adds to the sense that, while it's good to be in touch
> with the community, there isn't a lot of readiness to work on obscure
> things which don't affect a lot of people, and on hardware which isn't
> the future.
> 
> Eventually I might forward port enough to follow mainline kernels and
> tools.  But that's a substantial time sink too - don't underestimate
> it.  I'm told that mainline current 2.6 doesn't build let alone work
> on ARM-nommu (yet); that does not make forward-porting including local

I am down to 3 or 4 patches for mainline to work on arm non-mmu.
It could be made to work on almost all 2.6.x kernels with only a
few patches - and those have always been available from uclinux.org.
That is not a barrier. I am working out the wrinkles in these last
changes, they will get in sooner or later.


> drivers and architecture quirks sound like it will be quick.  If slow,
> it's not worth the effort to me alone.

>>> Just Google for "uclinux toolchain" and the top hits lead to very old
>>> releases, with bugs that have long been fixed elsewhere.  "uclinux arm
>>> toolchain" is no better.
>> The "fixed elsewhere" is the problem.  If everyone used the most current
>> release and worked through issues with the community, this problem would
>> go away.
> 
> I agree.  I would like to do that, and with a thriving, interested
> community I will.  I'm hoping linux-embedded@vger.kernel.org might
> help catalyse it - perhaps by creating the perception of it to people
> like me.
> 
> It only really works if lots of people do that together, and it seems
> to be particularly _not_ the culture in some segments of the embedded
> development community.
> 
> I base this on Google leading to lots of pages with old announcements
> of compilers, cross-compilation scripts and so on.  Why aren't those
> pages links to a few standard "good" places which are actively kept up
> to date?  It appears to be (or have been) a fragmented community - and
> so who to work with, that's a question.
> 
>>> Perhaps current versions (e.g. from Codesourcery?) are more dependable
>>> for embedded architectures, but I don't have the time to thoroughly
>>> test them, and my last experience warns me to be careful.
>> I know from experience, having worked for CodeSourcery that their
>> toolchains are exhaustively tested.  And any problems reported to their
>> public mailing lists are fixed, if legitimate.  These fixes are typically
>> submitted upstream to the FSF very quickly when the bugs exist in mainline
>> as well.
> 
> I know from experience that "bugs" are fixed, but feature requests
> like reliable C++, shared libraries and NPTL threads are long in
> coming to some architectures, if they ever will be before those
> architectures are no longer manufactured.
> 
>>> It seems people release tools, release patches, publish on an obscure
>>> web page, then forget about the page.  More authoritative-sounding
>>> uclinux web pages tend to be out of date.  Google isn't finding good
>>> current authorities in this area, which suggests the space is rather
>>> fragmented with people pulling in different directions and not working
>>> together enough to create stable, common places for these things.
>> But isn't the FSF repositories for the GNU Tools, and the uClinux projects
>> repositories the stable, common place for these things?
> 
> You can't even build an ARM-nommu executable with GNU tools from FSF!

Thats true of any of the non-mmu architectures that rely on
elf2flt. So arm is no different.

Like many open source projects uClinux isn't a finished "thing".
It is actively being developed. Not everything works, not everything
works well. The quality of what is there now depends very much
on the amount of effort input...


> It needs an extra tool.  I'm not even sure which version of that is
> the recommended one - the C one or the shell script both have issues
> outstanding.  By the way, neither work with some C++ programs on
> ARM-nommu: they crash with messages about unfixable relocations.  More
> often with GCC 4 than GCC 3.  See what I mean about things not being
> in great shape?

Well, what can I say, you are welcome to fix them :-)
Sometimes you need to be the one to step up and make
things better, especially if you want to use them :-)


>>> Contrast with kernel.org: everyone knows where to get a good working
>>> Linux kernel for the mainstream architectures, and the quality work
>>> tends to be quite good at reaching mainline there nowadays.
>> IMHO, the same is true for the GNU tool.
> 
> If that's true for ARM-nommu support, it has improved dramatically.  I
> will try again, in time, but prior experience tells me not to spend
> too long on it.  It's much better use of time to find some patched
> combination that is working well for someone else.  But who to
> ask/trust?  Even the pre-built toolchains don't always work right.
> (See my earlier mention of undebuggable thread problems in executables
> built with a googd-distributed.)

Regards
Greg


-- 
------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a Secure Computing Company      PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: Cross Compiler and loads of issues

[Greg Ungerer]

Jamie Lokier wrote:
> Bill Traynor wrote:
>> Maybe I'm being dense, but what's specifically wrong with the current
>> toolchain universe?
> 
> Back in ye olde days, you could download GCC and Binutils from
> gnu.org, configure for whatever is your architecture, and most times
> it just worked.
> 
> For some reason, that stopped a while ago, and you had to go to
> different places to get working basic tools.  And often, the place to
> go wasn't clear.  Different people advertised their "ARM toolchain",
> "m68k toolchain" etc.  and they were slightly different sets of
> patches on top of mainline tools.  Central authorities you might
> believe in existed, but they were always a few patches behind what you
> actually needed.
> 
> When I last needed a toolchain, Google led to confusing results, and I
> had to try more than one.  I still use mutiple GCC versions (from
> different people) to compile different programs for the same
> architecture: each one fails some things in a different way, including
> run-time failures in the precompiled toolchains.
> 
> Just Google for "uclinux toolchain" and the top hits lead to very old
> releases, with bugs that have long been fixed elsewhere.  "uclinux arm
> toolchain" is no better.

The uClinux arm toolchain case can be put down to noone
having an interest in actually doing the work. There has been
some work over the years, but no coordinated effort for quite
a while now.


> Perhaps current versions (e.g. from Codesourcery?) are more dependable
> for embedded architectures, but I don't have the time to thoroughly
> test them, and my last experience warns me to be careful.
> 
> It seems people release tools, release patches, publish on an obscure
> web page, then forget about the page.  More authoritative-sounding
> uclinux web pages tend to be out of date.  Google isn't finding good
> current authorities in this area, which suggests the space is rather
> fragmented with people pulling in different directions and not working
> together enough to create stable, common places for these things.

Yep, there is a lack of "working together" when it comes to the
uclinux tool chains. You can't force people to work together.

At the very least the packages on uclinux.org give you the build
instructions, and scripts used to build them. You can try to get
newer tool versions working (with fixes if required).


> Contrast with kernel.org: everyone knows where to get a good working
> Linux kernel for the mainstream architectures, and the quality work
> tends to be quite good at reaching mainline there nowadays.

I don't know that comparison exactly holds. kernel.org is source,
just like gnu.org is the source of gcc/binutils/etc. Are you not
talking about binary toolchain packages above?

Regards
Greg


-- 
------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a Secure Computing Company      PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: Cross Compiler and loads of issues

[Shaz]

On Fri, Jun 13, 2008 at 7:13 PM, Bill Traynor <wmat@naoi.ca> wrote:
>> It's nice to see we have so many options and related people and pros
>> to it are available around.
>>
>> IMO there should be some sort of effort to standardize the tool-chains
>> and build environments coherently with the kernel. I think its a prime
>> time to work around all the possibilities and standardize so that a
>> collaborative effort similar to Linux kernel can be set ahead. I think
>> thats the objective of this list too. Any plans yet sorted out?
>
> I don't understand what you mean by "standardize the tool-chains and build
> environments", can you provide specifics?  Similarly, what does "work
> around all the possibilities and standardize" mean?
>
Well, to put it clearly, there are many solutions to toolchains and
development kits but there must be work that can be stabilized by
 streamlining and sorting out similarities, especially with the kernel
 and libc. Simply, getting things done as smoothly as it would  have
 been with building linux from scratch.

 This might not make sense as I am a new comer to embedded world so
 pardon for any waste of time and thought process.

>> Maybe I'm being dense, but what's specifically wrong with the current
>>toolchain universe?
>>
>> On Fri, Jun 13, 2008 at 2:24 PM, Wookey <wookey@wookware.org> wrote:
>>>
>>> On 2008-06-12 22:52 +0500, Shaz wrote:
>>> > Hi,
>>> >
>>> > I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
>>> > felt like asking that is there one good way to get a cross compiler
>>> > work. I tried buildroot, scratchbox and even openMoko with
>>> > openEmbedded but all of them had lots of issues and don't know which
>>> > will be the best alternative.
>>>
>>> 'issues' are generally par for the course. It's a difficult problem.
>>> I've generally found buildroot the least-painful of the above, largely
>>> because it targets a relatively small section of the problem-space.
>>>
>>> For completeness I should also mention that Embedded Debian provides
>>> yet another option. I will not claim that it will have fewer issues
>>> than the above. http://www.emdebian.org/emdebian/intro.html
>>>
>>> I'm not sure if you are really asking about toolchains or build-systems?
>>> The former is fairly reliably solved. For a debian-based box just
>>> install pre-built cross toolchians from emdebian:
>>> http://www.emdebian.org/tools/crosstools.html
>>> (from i386, amd64, powerpc; to: nearly all debian-supported
>>> architectures, gcc3.3, 3.4, 4.1, 4.2, 4.3)
>>> http://www.emdebian.org/toolchains/search.php?section=devel gives more
>>> details.
>>>
>>> emdebian-tools also provides the debian equivalent of crosstool
>>> ('emchain' to build your own version of current current toolchain,
>>> should a suitable pre-built one not exist.
>>>
>>> For non-debian boxes other people seem to have listed the options so I
>>> won't repeat (and it's not my area of expertise).
>>>
>>> Wookey (Emdebian 'Chief bullshitter' - bias alert. The toolchains are
>>> great though, really.)
>>> --
>>> Principal hats:  Balloonz - Toby Churchill - Aleph One - Debian
>>> http://wookware.org/
>>
>>
>>
>> --
>> Shaz
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-embedded"
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>



-- 
Shaz

Firmware Linux (was Re: Cross Compiler and loads of issues)

[Rob Landley]

On Thursday 12 June 2008 12:52:44 Shaz wrote:
> Hi,
>
> I have been following "Re: [PATCH 0/1] Embedded Maintainer(s)" and
> felt like asking that is there one good way to get a cross compiler
> work. I tried buildroot, scratchbox and even openMoko with
> openEmbedded but all of them had lots of issues and don't know which
> will be the best alternative.

Did you try my FWL project? :)

http://landley.net/code/firmware

To build from source, go to http://landley.net/code/firmware/downloads and 
grab the latest version (firmware-0.4.0.tar.bz2).  Extract it, cd into it, 
and run "./build.sh".  That should list the available platforms, and then run 
with the platform you like as an argument (ala "./build.sh powerpc").

The build.sh wrapper runs the other scripts in sequence:
  ./download.sh - downloads source code if you haven't already got it.
  ./host-tools.sh - compiles stuff on the host platform, (mostly optional).
  ./cross-compiler.sh - build a cross compiler toolchain for your target.
  ./mini-native.sh - Compile a kernel and root filesystem for your target.
  ./package-mini-native.sh - Create an ext2 image out of mini-native.

The other interesting scripts in the directory are:
  ./forkbomb.sh - Build every target (--nofork in series, --fork in parallel).
      Calling with --fork is faster, but needs about 4 gigabytes of ram.
  ./run-emulator.sh - Boot one of the system images you just built under qemu.
      Also sets up the distcc trick (see below).

If you don't feel like compiling any of the above yourself, you can download 
prebuilt binary images from the same downloads link.  The cross-compiler 
directory has the prebuilt cross compilers for each target (for i686 and 
x86_64 hosts).  The mini-native directory has the prebuilt root filesystem 
images as tarballs.  And the system-image.sh directory has a tarball with 
that same mini-native root filesystem as an ext2 image, a kernel for qemu, 
and shell scripts to invoke qemu appropriately for each one:

  ./run-emulator.sh - Run qemu, booting to a shell prompt.
  ./run-with-home.sh - Calls run-emulator with a second hard drive image
      hooked up as /dev/hdb and mounted on /home.  (If you haven't got an
      hdb.img in the directory it'll create an empty 2 gigabyte image and
      format it ext3.)
  ./run-with-distcc.sh - Calls run-with-home with the distcc trick set up.

The reason "mini-native" is called that is because it's a minimal native 
development environment.  It contains gcc, binutils, make, linux kernel 
headers, uClibc for your C library, and standard susv3 command line tools 
(provided by the busybox and toybox packages).  This is theoretically enough 
to build the whole of Linux From Scratch (http://www.linuxfromscratch.org) or 
bootstrap your way up to being able to natively build gentoo or debian using 
their package management systems.  (If you don't _want_ development tools in 
your root filesystem, set the environment variable "BUILD_SHORT=1" before 
running ./mini-native.sh.  That'll also move it out of /tools and up to the 
top level.)

In practice, the environment is still missing seven commands (bzip2, find, 
install, od, sort, diff, and wget) needed to rebuild itself under itself.  
(The busybox versions were either missing or had a bug.)  I'm working on that 
for the next release.

The distcc trick is for speeding up native builds by using distcc to call out 
to the cross compiler.  Running the cross compiler on the host system is 
faster, but cross compiling is very brittle.  This approach does a native 
build under the emulator, but escapes the emulator for the actual compiling 
part.  I did a quick bench test compling make 3.81 (I had it lying around) 
and it sped up the actual compile by a factor of 7.  (Alas, it didn't speed 
up the "./configure" part at all, just the "make" part.)

The ./emulator-build.sh script sets up the distcc trick using the files left 
in the "build" directory after you build it yourself.  (The cross compiler is 
left in there, and the kernel and ext2 system images that got tarred up are 
also left in there.)  The ./run-with-distcc.sh script does it for 
cross-compiler and system-image tarballs.  (It needs one argument: You have 
to tell it the path where you extracted the cross-compiler tarball.)

> I also went through the material provided freely by Free Electron but
> still I am not successful to build a custom kernel. Next I am trying
> MontaVista's kit. I just wish I don't get lost.

I'm happy to answer any questions about the stuff I did... :)

> Anyways, I liked the idea of Qemu based cross compiler. Is it possible
> for the inexperienced to get it working and emulate the exact model
> and devices.

That's what I'm trying to do.  I've got armv4l, armv5l, mips (big endian), 
mipsel (little endian), powerpc (a prep variant), i586, i686, and x86_64 
working.  They all work fine.  Run "./smoketest.sh $ARCH" on each $ARCH after 
building it to see the sucker boot up and compile "hello world" using distcc.  
That means qemu has bootable kernel, serial port, emulates a hard drive, 
uClibc is configured right, there's a working native toolchain, working cross 
toolchain, working virtual network...

I've also got sh4 building (but qemu doesn't quite have enough support to boot 
it yet: it can't emulate any boards with a hard drive attached).  And I've 
got sparc building and mostly booting, but it hangs trying to read 
from /dev/console for reasons I haven't been motivated to track down because 
I don't really know anyone still using sparc.  And I've got most of an m68k 
build config together but gcc is giving me an "internal compiler error" 
trying to build uClibc.  I've also poked at Alpha and blackfin a bit, but 
again lack of sufficient qemu support takes some of the fun out of it.

Oh, and the powerpc emulated prep board/kernel don't agree how to shutdown the 
power to the virtual board, so the qemu process doesn't exit.  You have to 
kill it.  :P

I can has TODO items...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: Firmware Linux (was Re: Cross Compiler and loads of issues)

[Enrico Weigelt]

* Rob Landley <rob@landley.net> schrieb:

> Did you try my FWL project? :)
> 
> http://landley.net/code/firmware

hmm, doesnt look like supporting sysroot ...


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: Firmware Linux (was Re: Cross Compiler and loads of issues)

[Rob Landley]

On Sunday 15 June 2008 23:38:12 Enrico Weigelt wrote:
> * Rob Landley <rob@landley.net> schrieb:
> > Did you try my FWL project? :)
> >
> > http://landley.net/code/firmware
>
> hmm, doesnt look like supporting sysroot ...

It doesn't use sysroot.  It makes the compiler relocatable using an updated 
version of the old uClibc wrapper script, which rewrites each glibc command 
line to start with --nostinc --nostdlib and then adds the correct paths back 
in from the ground up.

Have you ever run gcc under strace?  Or worse, looked at the gcc path logic 
source code?  It's a mess.  It gets paths from ./configure options, it gets 
paths from spec files, it adds hardwired paths in the C source code, it 
checks enviroment variables to get more paths, and this isn't counting the 
paths you tell it on the command line.  Every time they gave up on the 
previous approach because it was obviously unworkable, they NEVER REMOVED 
ANYTHING.  They just added yet another layer on top of it, falling back to 
the previous one each time.  It never occurred to them that people would want 
to make it NOT check paths like /usr/include.  They just keep piling on more 
and more, appending to a big vararray and never removing anything.

Do you know how sysroot is implemented?  It still hardwires absolute paths 
into the binary, but then it does a string compare with the start of the 
hardwired path so it knows how much to trim off when substituting another 
path instead.

I once tried to work up a patch to remove the obsolete or clearly defective 
parts of the gcc path logic, but when my patch got large than 10,000 lines I 
gave up and went to a wrapper script.  The only way to make the gcc path 
logic reliable is to NOT USE IT.  Hence the wrapper script to tell it to 
ignore all the paths it _thinks_ it knows, and check in exactly these places 
(and only those places) instead.  Then you can run the result under strace 
without wanting to throw up quite so much.

> cu

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.