Re: building Rootfs

Re: building Rootfs

[Linus Walleij]

OK Robert wrote this on ARM-Linux mailinglist but the discussion
belongs on linux-embedded, so I'm moving it there, the suggested
ptxdist ML seems inapropriate for the entire embedded
community.

2008/7/7 Robert Schwebel <r.schwebel@pengutronix.de>:

> On Mon, Jul 07, 2008 at 06:01:51PM +0800, Eric Miao wrote:
>> or start from buildroot, scratchbox and openembedded as a tool, you've
>> really got a lot to learn :-)
>
> Or ptxdist, or course :-)

Robert, can you brief us of how ptxdist fits together with
OpenEmbedded? What does these two projects actually
share? Where do they do similar things in parallel for
example?

I'm trying - real hard - to get an idea of how people out
there prefer to build their root filesystems in cross-compiled
environments.

Rob Landley recently wrote up a small blurb on why native
compilation is the way to go, and a small roadmap on how
he intended to get there using e.g. Qemu and Firmware Linux,
c.f: http://www.landley.net/code/firmware/about.html
which finally won me over to that line of thinking.
Debian and friends obviously go this way now.

However, when it comes to the widespread and much
fragmented ways of cross-compiling a rootfs, including the
stuff put together by MontaVista, WR and all those animals
in the forest, obviously based on RPM (build systems I
haven't put my hands inside, since they are proprietary)
there seems to be very little consensus. ptxdist stands out
but do you get a lot of outside contributions for it? As it
looks it seems you're running it yourself. (Beware: I
haven't looked close.)

What else is there out there for rootfs, really? A hack
from every embedded company there is? I'm more after
what people actually *use* and what is community driven
here, not so much opinions on what is best (which will
probably be the unwanted side effect of this mail
anyway...)

Linus

Re: building Rootfs

[Robert Schwebel]

On Mon, Jul 07, 2008 at 10:34:11PM +0200, Linus Walleij wrote:
> Robert, can you brief us of how ptxdist fits together with
> OpenEmbedded? What does these two projects actually share? Where do
> they do similar things in parallel for example?

They are both build systems for userlands, or whole embedded linux
systems. I cannot speak for OE as I don't really know it in detail.

The idea behind ptxdist is executable documentation. If you use Open
Source software for business critical industrial applications, having
control over the source is an important thing. So our customers are
usually able to reproduce their root filesystems with one "ptxdist go"
directly from the sources. Our design criteria are

- things are highly configurable
- you have to find out what was done why
- going the mainline way where ever possible
- use proven technology (ptxdist is written in bash & gnu make)

> I'm trying - real hard - to get an idea of how people out there prefer
> to build their root filesystems in cross-compiled environments.

Well, I still havn't seen a real argument against it. At least not for
our usecases.

> Rob Landley recently wrote up a small blurb on why native
> compilation is the way to go, and a small roadmap on how
> he intended to get there using e.g. Qemu and Firmware Linux,
> c.f: http://www.landley.net/code/firmware/about.html
> which finally won me over to that line of thinking.
> Debian and friends obviously go this way now.

Cool, so what do we do for platforms like Blackfin, or any of the random
ARM processors out there, write qemu support first? Sorry, you cannot
really suggest this. Not with Qemu being in such a crude state that it
can not even be built with modern compilers.

Being able to be cross compiled is actually a *feature* of unix
software, be it autotoolized or not. And, in reality, the problems which
usually come up in these 'build native' discussions are very often
academic.

> However, when it comes to the widespread and much fragmented ways of
> cross-compiling a rootfs, including the stuff put together by
> MontaVista, WR and all those animals in the forest, obviously based on
> RPM (build systems I haven't put my hands inside, since they are
> proprietary) there seems to be very little consensus.

Well, the consensus is usually configure && make && make install. If
people just wouldn't assume that they know it better than autotools ...

But don't let us start a flamewar in this direction here. It doesn't
bring us anywhere.

> ptxdist stands out but do you get a lot of outside contributions for
> it? As it looks it seems you're running it yourself. (Beware: I
> haven't looked close.)

Oh, we have quite a bunch of contributions from the outside, and almost
200 people on the mailing list. It goes up and down, there are times
where we get more contributions from the customers, then from the
community, but in general it works really good.

Note that it is a tool to solve *our* problems (which consist of making
reproducable embedded linux systems for our customers, while being able
to fix problems as they arise). And it solves them very, very well.

> What else is there out there for rootfs, really? A hack from every
> embedded company there is?

PTXdist doesn't invent something new, it just exercises the usual
configure && make && make install canon, in a reproducable way. Nothing
more and nothing less. It is completely open, being GPL, so you can
freely choose if you accept realizing it being on the way to world
domination or not :-)

It's not even focussed on building root filesystems. We also use it to
build cross compilers and well defined Eclipse installations.

> I'm more after what people actually *use* and what is community driven
> here, not so much opinions on what is best (which will probably be the
> unwanted side effect of this mail anyway...)

PTXdist is in use and community driven, so it migth be worth a look :-)

rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Re: building Rootfs

[Linus Walleij]

2008/7/7 Robert Schwebel <r.schwebel@pengutronix.de>:

> On Mon, Jul 07, 2008 at 10:34:11PM +0200, Linus Walleij wrote:

>> Robert, can you brief us of how ptxdist fits together with
>> OpenEmbedded? What does these two projects actually share? Where do
>> they do similar things in parallel for example?
>
> They are both build systems for userlands, or whole embedded linux
> systems. I cannot speak for OE as I don't really know it in detail.

Anyone have experience with both? I'd really like to have them
contrasted and, well you know, cross-firtualized.

> The idea behind ptxdist is executable documentation.

I really like ptxdist, especially it is beautiful that you have
based it around just bash and make plus some crosscompiler
of choice. I just have some trouble getting it to do what
I want. I'll hopefully get there. This is not meant as any generic
complaining. I'm trying to figure out if ptxdist or openembedded
is the best starting point.

I have a situation where the build system would be on top
of some SCM like git, CVS or whatever and packages being
checked out and worked on by several people daily.
This seems to clash with ptxdist design goals,
for example:

ptxdist seems
* hardcoded to run in a users home directory
  with projects in $HOME/foo (stopped me
  right now, perhaps it can easily be fixed or is
  already part of ptxdist 2?)
* assume only one person builds rootfs. So
  it doesn't really want the project tree to be used
  or modified by more than one person at the time.
  (Same as first more or less.)
* not store source code, patches and config plus my
  local additions in one place. For most other package
  systems e.g. RPM there is
  /foo/foo.spec
  /foo/foo-1.0.tar.gz
  /foo/foo-patch1.diff
  /foo/foo-patch2.diff
  /foo/my-foo-config.file

In ptxdist these seem to be spread out, some in the installation
of ptxdist itself in say /usr/lib/ptxdist/patches, /usr/lib/ptxdist/...
then in my project space $HOME/project/... etc.

I believe at archivicing time everything including ptxdist itself
is zipped up and archived on one storage media. But I need to
keep all files relating to a package in one place while developing,
and living like that for a long time. Since /usr/lib/ptxdist isn't even
being backed up normally, only NFS mounts like /home, I then
need to move the entire ptxdist into the SCM, which it clearly
doesn't like without a lot of hacking, plus, how would I
upgrade it?

OK this is a bit picky. And perhaps I can fix it all and merge
back. Just trying to decide.

Linus

Re: building Rootfs

[Robert Schwebel]

On Wed, Jul 09, 2008 at 02:14:04AM +0200, Linus Walleij wrote:
> > They are both build systems for userlands, or whole embedded linux
> > systems. I cannot speak for OE as I don't really know it in detail.
>
> Anyone have experience with both? I'd really like to have them
> contrasted and, well you know, cross-firtualized.

When we started with ptxdist, OE was not there. When I last tried it, we
still saw the following advantages in our design:

- standard unix technology (make+bash) instead of python did fit our
  skills better (yes, a very personal argument)

- we have a "project" pattern: users have their "project workspace"
  which can for example be revisioned in the user's own scm.

- ptxdist can be installed as a tool (to be installed with configure &&
  make && make install) and contains anything you need, besides the
  "project". With OE, we had to checkout bit repositories.

- ptxdist has a clear revision policy, with stable and unstable
  development branches. There are numbered releases available.

- ptxdist has full configurability; everything can be configured with
  kconfig menues.

Other than that, the OE people have very good ideas and found similar
problems and solutions as the ptxdist team did. They have a similar
patch stack concept which makes it easy to share ideas.

> I really like ptxdist, especially it is beautiful that you have based
> it around just bash and make plus some crosscompiler of choice.

Yup :-)

> I just have some trouble getting it to do what I want. I'll hopefully
> get there. This is not meant as any generic complaining. I'm trying to
> figure out if ptxdist or openembedded is the best starting point.

So if you see problems, just post your ptxconfig file on the ptxdist
mailing list. We will be glad to discuss your problems there.

> I have a situation where the build system would be on top of some SCM
> like git, CVS or whatever and packages being checked out and worked on
> by several people daily.

You can overwrite almost everything in the project that is defined in
ptxdist; for example, if you have a rule set rules/foobar.make and a
corresponding menu rules/foobar.in in ptxdist, you can also have these
files in the project workspace and they overrule the ones which come
from the base package.

So what you can do is:

- have your project workspace, like this:
  /home/you/my_project/ptxconfig
  /home/you/my_project/rules/my_packet.in
  /home/you/my_project/rules/my_packet.make

- If several people shall work on the "local" packages in rules/, just
  put the whole workspace into your SCM. That works pretty well in real
  life; we have done big projects that way, where several ten people
  worked together in a project team, synchronized via SVN.

- You can also make use of svn externals: for example, we have a shared
  "technology pool" here at Pengutronix, with packages that are not
  ready for mainline but need to be shared. We just bring them into a
  project via svn:externals.

- Imagine team member A works on package foobar, has tested it and
  released foobar-1.0.0 somwhere in the repository. He then changes
  rules/foobar.make to contain the revision number 1.0.0. Then he
  continues developing, changing the revision to -trunk or whatever he
  uses internally, but only on his workspace, without checking in. This
  way he can do all the development, until the point that he has
  released 1.0.1. Then he just checks in the changed .make file with the
  new revision and everybody else sees the new and tested packet.

What does not work yet out-of-the-box is that one team develops for
example a base platform, which is then made available in binary form,
and everyone else gets just the binary packets foo_x.y.z_arch.ipk
instead of compiling from scratch.

We have ideas how to realize that, as it is a common pattern in our
projects. Will be a ptxdist-2.x issue.

> ptxdist seems
> * hardcoded to run in a users home directory
>   with projects in $HOME/foo (stopped me
>   right now, perhaps it can easily be fixed or is
>   already part of ptxdist 2?)

It is already part of ptxdist-1.0.x. You can put your project workspace
whereever you want.

> * assume only one person builds rootfs. So
>   it doesn't really want the project tree to be used
>   or modified by more than one person at the time.
>   (Same as first more or less.)

If different people are working on it, everyone has his own working
copy. It's not different from other revisioned projects.

> * not store source code, patches and config plus my
>   local additions in one place. For most other package
>   systems e.g. RPM there is
>   /foo/foo.spec
>   /foo/foo-1.0.tar.gz
>   /foo/foo-patch1.diff
>   /foo/foo-patch2.diff
>   /foo/my-foo-config.file
> 
> In ptxdist these seem to be spread out, some in the installation
> of ptxdist itself in say /usr/lib/ptxdist/patches, /usr/lib/ptxdist/...
> then in my project space $HOME/project/... etc.

Think of it as object orientation, written in make :-) For each packet
(let's say bash), ptxdist brings it's own information with it:

rules/bash.make			rule set (what is to be done)
rules/bash.in			menu definition (kconfig)
patches/bash-x.y/generic/	patch series (quilt format, canonical
				patches, following kernel rules)

PTXdist brings a "set" of such packages with each installation of the
tool itself (i.e. ptxdist-1.0.2 has it's own packet set).

You can overwrite packages (make, in and patches) by simply putting them
into the project. This makes it possible to change things in single
packets, without the need to switch to a new ptxdist version.

> I believe at archivicing time everything including ptxdist itself
> is zipped up and archived on one storage media. But I need to
> keep all files relating to a package in one place while developing,
> and living like that for a long time. Since /usr/lib/ptxdist isn't even
> being backed up normally, only NFS mounts like /home, I then
> need to move the entire ptxdist into the SCM, which it clearly
> doesn't like without a lot of hacking, plus, how would I
> upgrade it?

Archiving is pretty easy: all you need is:

- the ptxdist tarballs you have installed from
- all the source tarballs which have been downloaded on the first run of
  the "get" stages
- your project workspace

It was *desinged* to be easily archived, so this is exactly what has to
be done.

HTH,
rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

RE: building Rootfs

[Haller, John H (John)]

> 
> What else is there out there for rootfs, really? A hack
> from every embedded company there is? I'm more after
> what people actually *use* and what is community driven
> here, not so much opinions on what is best (which will
> probably be the unwanted side effect of this mail
> anyway...)
> 
> Linus

There are three issues in building an embedded cross-filesystem:
1. Identification of the files to put into the filesystem.
2. Stripping or extracting debug symbols from the filesystem.
3. Getting files into the filesystem when the development
system does not allow root access.

The first item is made a little easier by RPM, as it groups
files together into packages, and package dependencies are
quite often properly declared. However, packages sometimes
have dependencies on another package when only part of the
package may be required. Another problem is with documentation
files (usually unwanted on an embedded system), and locales
which are huge and generally unwanted.

The second item can be tricky, as one doesn't want to
strip debug symbols every time a filesystem is built, but
the debug symbols are generally needed for cross debugging.
debugedit from the RPM package does a nice job of this,
separating the debug symbols from the executable, but
it is best to do this when an cross executable is installed
on the host system, rather than at filesystem build time.

The third item is easy for an initramfs, but not well handled
by existing tools for an initrd (although I haven't looked
recently). One possibility is to modify a mkfs program, such
as mke2fs, for the filesystem of choice to accept proto
files, like the traditional Unix mkfs does. [Google mkfs proto
for some examples]. I'm not sure if ptxdist supports both
initramfs and initrd. If you do have root access, this is
all much simpler, as an initrd can be locally mounted
and populated.

One other item that pops up sometimes is post-install scripts
that are associated with some packages. These scripts run in
the target filesystem, and there is no good way to run them
in the host environment. This doesn't happen too often, so
it can usually be handled by hand as an exception. One example
of this type of item is installing the symbolic links in the
right place for init scripts. Another option would be to
capture the scripts and run them during boot, but this would
slow boot times.

Using the tools from an embedded vendor has two problems -
they can change from release to release, and they are proprietary
tools that lock you into the vendor. But, they may provide
a GUI, if you are into graphical presentations.



John Haller

Re: building Rootfs

[Robert Schwebel]

On Mon, Jul 07, 2008 at 06:24:48PM -0500, Haller, John H (John) wrote:
> There are three issues in building an embedded cross-filesystem:
> 1. Identification of the files to put into the filesystem.
> 2. Stripping or extracting debug symbols from the filesystem.
> 3. Getting files into the filesystem when the development
> system does not allow root access.

Ack. In PTXdist, we also cover

4. what to put into the initial image on deployment (we had what you
   describe in 3 above but split that up into "what to build" vs. "what
   to install" in ptxdist-2.x).

5. how to push the image into the device (managing and writing
   complicated u-boot environments and pushing it into the device
   automatically, via serial+ethernet)

6. automatic test suites against embedded targets

> The first item is made a little easier by RPM, as it groups files
> together into packages, and package dependencies are quite often
> properly declared. However, packages sometimes have dependencies on
> another package when only part of the package may be required. Another
> problem is with documentation files (usually unwanted on an embedded
> system), and locales which are huge and generally unwanted.

That was the reason why we went with ipkg; the meta data of rpm and deb
are too large to be hold on an embedded system, but we want software
updatability. However, if someone needs any sort of Carrier Grade
Blubberblabber, we could easily add rpm and/or deb :-)

> The second item can be tricky, as one doesn't want to strip debug
> symbols every time a filesystem is built, but the debug symbols are
> generally needed for cross debugging.

Yup; ptxdist solves this by having root/ and root-debug/, where the
first one can be booted via NFS or flash, and the second one contains
all the debug symbols. Cross debugging works that way, even with Eclipse
(for the people who deny seeing the pure elegance and shinyness of vim
and commandline gdb).

> The third item is easy for an initramfs, but not well handled by
> existing tools for an initrd (although I haven't looked recently). One
> possibility is to modify a mkfs program, such as mke2fs, for the
> filesystem of choice to accept proto files, like the traditional Unix
> mkfs does. [Google mkfs proto for some examples]. I'm not sure if
> ptxdist supports both initramfs and initrd. If you do have root
> access, this is all much simpler, as an initrd can be locally mounted
> and populated.

I'm not sure as well; the parts for initramfs are there, but we don't
use it here at Pengutronix, because all of our current customer systems
have normal flash root filesystems; we once had a system with IXP425
that needed a userspace firmware for nfsroot, but that's the only case I
can remember.

However, if somebody needs improvement, he is free to send patches...

> One other item that pops up sometimes is post-install scripts
> that are associated with some packages. These scripts run in
> the target filesystem, and there is no good way to run them
> in the host environment.

This is already solved :-) We have pre/post scripts, this way:

rules/foobar.make
rules/foobar.preinst
rules/foobar.postinst
...

In the scripts you can check if DESTDIR is set; if yes, you are in the
host environment (development system tries to build a root fs from the
ipkgs); if not, you are on the target (something does 'ipkg install').

With that mechanics, you can for example decide that on the target you
need to stop a service first, then update, then start it again.
Everything but update isn't required if only an image is being made.

> This doesn't happen too often, so it can usually be handled by hand as
> an exception. One example of this type of item is installing the
> symbolic links in the right place for init scripts.

Yup, that can be done with the mechanics above.

> Using the tools from an embedded vendor has two problems -
> they can change from release to release,

We have a stable ptxdist-1.x series with long term bugfix maintenance
for that ... even this stable series is completely open source, for
everyone's download pleasure.

> and they are proprietary tools that lock you into the vendor.

No need to do so. Everything in ptxdist is GPL. If somebody wants to be
locked to Pengutronix, we feel better if he does so because he has the
warm and cuddly feeling that we solve his problems best ... which, by
the way, doesn't work too bad.

> But, they may provide a GUI, if you are into graphical presentations.

Well, if someone absolutely needs a GUI, he can use gvim¹ :-)

rsc

¹ Or even Eclipse & vmware, if your software department has a better
  feeling while using buzzword technology.
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Re: building Rootfs

[Wolfgang Denk]

In message <B78930B8E34A824993912C39C24569E60141D874@ILEXC3U01.ndc.lucent.com> you wrote:
>
> There are three issues in building an embedded cross-filesystem:
> 1. Identification of the files to put into the filesystem.
> 2. Stripping or extracting debug symbols from the filesystem.
> 3. Getting files into the filesystem when the development
> system does not allow root access.

3. is a non-issue for most common file systems.

> The third item is easy for an initramfs, but not well handled
> by existing tools for an initrd (although I haven't looked
> recently). One possibility is to modify a mkfs program, such

genext2fs has been around for a long, long time.

And there are similar tools for other file system types  like  cramfs
or jffs2 ...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
The human race is a race of cowards; and I am not  only  marching  in
that procession but carrying a banner.                   - Mark Twain

Re: building Rootfs

[Linus Walleij]

2008/7/8 Wolfgang Denk <wd@denx.de>:

>> 3. Getting files into the filesystem when the development
>> system does not allow root access.

> 3. is a non-issue for most common file systems.

The only one thing I ever ran into trouble with was device nodes,
these cannot be reproduced any way, not even with fakeroot
environments, just in scratchbox, which in turn needs you to be root.

initramfs etc can handle it with special description files. But if
you want to cook up say a .tar file of your rootfs, you're pretty
much lost AFAIK.

The good thing is that you don't need the device nodes
if you have udev, I think the kernel wants /dev/console
and a few more at boot but actually it survives just fine
without them.

The rest is possible to create with fakeroot and clever
scripting if I'm not mistaken.

If you know some way of sneaking a device node into a
.tar file created ENTIRELY running as a regular user,
tell me!

Linus

Re: building Rootfs

[David VomLehn]

Linus Walleij wrote:
> If you know some way of sneaking a device node into a
> .tar file created ENTIRELY running as a regular user,
> tell me!

We do this all the time. We have a very minimal root filesystem and aren't 
currently using any of the frameworks for building root filesystems, so 
everything is done with make. In general, we create tar files for each piece. The 
piece that has device nodes has a make file with something like:

	all:
		fakeroot make fakeroot-package

	fakeroot-package:
		mkdir -p image
		mknod image/dev/null c 1 3
		mknod image/dev/random c 1 8
		tar -C image -czf image.tgz .

We ran into some issues when we separated running mknod and tar into separate 
fakeroot invocations because there is a bug in fakeroot with exporting and 
importing its state. Fortunately, doing all of this under a single fakeroot 
command works just fine. When we untar the tar files for all of the pieces to 
create a root filesystem image and then run mkcramfs to create a CRAMFS image, we 
do all commands under a single invocation of fakeroot.

No stage of the build requires running as root.
-- 
David VomLehn, dvomlehn@cisco.com




     - - - - -                              Cisco                            - - - - -         
This e-mail and any attachments may contain information which is confidential, 
proprietary, privileged or otherwise protected by law. The information is solely 
intended for the named addressee (or a person responsible for delivering it to 
the addressee). If you are not the intended recipient of this message, you are 
not authorized to read, print, retain, copy or disseminate this message or any 
part of it. If you have received this e-mail in error, please notify the sender 
immediately by return e-mail and delete it from your computer.

Re: building Rootfs

[Robert Schwebel]

On Tue, Jul 08, 2008 at 05:52:34PM -0700, David VomLehn wrote:
> We do this all the time. We have a very minimal root filesystem and 
> aren't currently using any of the frameworks for building root 
> filesystems, so everything is done with make. In general, we create tar 
> files for each piece. The piece that has device nodes has a make file 
> with something like:
>
> 	all:
> 		fakeroot make fakeroot-package
>
> 	fakeroot-package:
> 		mkdir -p image
> 		mknod image/dev/null c 1 3
> 		mknod image/dev/random c 1 8
> 		tar -C image -czf image.tgz .

That does work fine when packaging the rootfs on the host. It was the
reason why we went the ipkg way: you can put device nodes into them.

> We ran into some issues when we separated running mknod and tar into 
> separate fakeroot invocations because there is a bug in fakeroot with 
> exporting and importing its state. Fortunately, doing all of this under a 
> single fakeroot command works just fine. When we untar the tar files for 
> all of the pieces to create a root filesystem image and then run mkcramfs 
> to create a CRAMFS image, we do all commands under a single invocation of 
> fakeroot.
>
> No stage of the build requires running as root.

The only stage we need root privileges in PTXdist is when creating a
root filesystem which can be booted via nfsroot (which is common during
development). In that case, you *need* /dev/console and /dev/null, which
cannot be created by the user.

We have solved that by using sudo there: the automatic build asks for
the user password, but with a timeout. So in interactive builds, you can
just enter the password and have a rootfs ready to be booted, and it
still works with automatic overnight builds.

rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Re: building Rootfs

[Wolfgang Denk]

In message <63386a3d0807081653t5e19cae2vd35c03de4bec7fa8@mail.gmail.com> you wrote:
> 
> >> 3. Getting files into the filesystem when the development
> >> system does not allow root access.
> 
> > 3. is a non-issue for most common file systems.
> 
> The only one thing I ever ran into trouble with was device nodes,

No, this is not a problem.

> these cannot be reproduced any way, not even with fakeroot

Most tools support a "device table", i. e. a plain text file
describing needed device nodes, owners, permissions etc., so no root
permissins are needed.

For details please see
http://www.denx.de/wiki/view/DULG/RootFileSystemOnARamdisk

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Beware of the Turing Tar-pit in  which  everything  is  possible  but
nothing of interest is easy.

Re: building Rootfs

[Robert Schwebel]

On Wed, Jul 09, 2008 at 01:53:46AM +0200, Linus Walleij wrote:
> 2008/7/8 Wolfgang Denk <wd@denx.de>:
> 
> >> 3. Getting files into the filesystem when the development
> >> system does not allow root access.
> 
> > 3. is a non-issue for most common file systems.
> 
> The only one thing I ever ran into trouble with was device nodes,
> these cannot be reproduced any way, not even with fakeroot
> environments, just in scratchbox, which in turn needs you to be root.

You can create an image with fakeroot which contains the device nodes,
then flash it into the device as a normal user and boot it there.

> initramfs etc can handle it with special description files. But if
> you want to cook up say a .tar file of your rootfs, you're pretty
> much lost AFAIK.

Right, but it's no real issue.

> The good thing is that you don't need the device nodes
> if you have udev, I think the kernel wants /dev/console
> and a few more at boot but actually it survives just fine
> without them.

Udev is pretty slow, so we have both possibilities in ptxdist. Users
have a choice if they want to be as "normal" as possible (udev) or if
they have faster requirements. For boot time optimized systems, we have
even done mixed systems, with static device nodes in the (quick) boot
phase and udev later on.

> If you know some way of sneaking a device node into a .tar file
> created ENTIRELY running as a regular user, tell me!

The question is: why do you need a tar file?

Hmm, I'm wondering if this discussion isn't off-topic here, as it is a
kernel mailing list. But I know no other list anyway, so if nobody
complains ...

rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Re: building Rootfs

[Peter Korsgaard]

>>>>> "Linus" == Linus Walleij <linus.ml.walleij@gmail.com> writes:

 Linus> If you know some way of sneaking a device node into a
 Linus> .tar file created ENTIRELY running as a regular user,
 Linus> tell me!

You just mentioned it yourself - fakeroot.

-- 
Bye, Peter Korsgaard

Re: building Rootfs

[Marek Skuczynski]

Haller, John H (John) pisze:
>> What else is there out there for rootfs, really? A hack
>> from every embedded company there is? I'm more after
>> what people actually *use* and what is community driven
>> here, not so much opinions on what is best (which will
>> probably be the unwanted side effect of this mail
>> anyway...)
>>
>> Linus
> 
> There are three issues in building an embedded cross-filesystem:
[--CUT--]
> 3. Getting files into the filesystem when the development
> system does not allow root access.

Couple years ago, I found a simple solution for this issue. The 
filesystem image is described in the initramfs_list file format. That is 
  easy to modify (adding new nodes, fixing paths, etc).
The final image building scenario is as follows:

cat initramfs_list | check_fs_integrity | mk<fs>image > image_file

Of course the initramfs_list file is cooked up dynamically basis on 
configuration of kernel, drivers and etc.

mareksk