initramfs size limitation

initramfs size limitation

[Phil Wilshire]

Hi All,

Firstly, thanks for this list I hope we can get some interesting things 
happening.

I have a question regarding initramfs and tmpfs in a NOMMU environment.

I hope this is the right place and the right sort of question.

I work closely with the Blackfin systems and they have now integrated 
the initramfs generation into their system build. The result is great
the root fs is ready to run from the page cache.

There is one problem that I can see that may be more serious for 
embedded users.
As far as I can tell the initramfs filesystem is not restricted in size.
You can keep writing files until it uses all available memory.

It should be possible to mount (or rather remount) the initramfs read 
only and use tmpfs for a restricted size read/write file system.

The trouble is that the size option for tmpfs is removed for NOMMU systems.

Has anyone else seem this problem ?
I dont think the fixes are complicated but has any one attempted to fix it ?
Would the mainline kernel be receptive to such modifications ?


Regards
   Phil Wilshire

Re: initramfs size limitation

[Rob Landley]

On Saturday 31 May 2008 07:44:14 Phil Wilshire wrote:
> Hi All,
>
> Firstly, thanks for this list I hope we can get some interesting things
> happening.
>
> I have a question regarding initramfs and tmpfs in a NOMMU environment.

/me pleads the fifth.

> I hope this is the right place and the right sort of question.
>
> I work closely with the Blackfin systems and they have now integrated
> the initramfs generation into their system build. The result is great
> the root fs is ready to run from the page cache.

Is it possible to get blackfin working with a vanilla gcc release yet, or do 
you still need out-of-tree patches?  (I have a blackfin board I got at OLS, 
but it needs a toolchain I can't reproduce.)

All my builds these days are based on http://landley.net/code/firmware (and in 
fact it would be really nice if qemu grew blackfin support because messing 
with real hardware is a pain while using a laptop at coffee shops).

> There is one problem that I can see that may be more serious for
> embedded users.
> As far as I can tell the initramfs filesystem is not restricted in size.
> You can keep writing files until it uses all available memory.

Yup.  There have intermittently been patches to make rootfs be tmpfs instead 
of ramfs, the most recent of which I remember was:
http://lkml.org/lkml/2007/7/13/354

If they ever got merged, I didn't hear about it.  People keep bringing up 
vague handwaving about potential problems with memory management setup 
sequencing, since tmpfs potentially interacts with swap and that's not set up 
yet when tmpfs gets mounted.  (Then again no swap partitions are mounted at 
that point either.)  I'm unaware of anybody who tried it reporting any actual 
problems, but oh well...

> ? Would the mainline kernel be receptive to such modifications ?

Post a patch, get a signed-off-by David Woodhouse (in his role as embedded 
maintainer), and submit to Andrew Morton's tree.  That way, at each point, 
you at least know who owes you a response (even if it's "no").

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: initramfs size limitation

[Mike Frysinger]

On Sun, Jun 1, 2008 at 8:03 PM, Rob Landley wrote:
> On Saturday 31 May 2008 07:44:14 Phil Wilshire wrote:
>> I hope this is the right place and the right sort of question.
>>
>> I work closely with the Blackfin systems and they have now integrated
>> the initramfs generation into their system build. The result is great
>> the root fs is ready to run from the page cache.
>
> Is it possible to get blackfin working with a vanilla gcc release yet, or do
> you still need out-of-tree patches?  (I have a blackfin board I got at OLS,
> but it needs a toolchain I can't reproduce.)

there's plenty of usable binaries available

> fact it would be really nice if qemu grew blackfin support because messing

i imagine it would be ... too bad qemu lacks real documentation

>> There is one problem that I can see that may be more serious for
>> embedded users.
>> As far as I can tell the initramfs filesystem is not restricted in size.
>> You can keep writing files until it uses all available memory.
>
> Yup.  There have intermittently been patches to make rootfs be tmpfs instead
> of ramfs, the most recent of which I remember was:
> http://lkml.org/lkml/2007/7/13/354

it wouldnt matter.  tmpfs on NOMMU doesnt support any of the options
like MMU.  look at mm/tiny-shmem.c and init/Kconfig.
-mike

Re: initramfs size limitation

[Bryan Wu]

On Mon, Jun 2, 2008 at 8:25 AM, Mike Frysinger <vapier.adi@gmail.com> wrote:
> On Sun, Jun 1, 2008 at 8:03 PM, Rob Landley wrote:
>> On Saturday 31 May 2008 07:44:14 Phil Wilshire wrote:
>>> I hope this is the right place and the right sort of question.
>>>
>>> I work closely with the Blackfin systems and they have now integrated
>>> the initramfs generation into their system build. The result is great
>>> the root fs is ready to run from the page cache.
>>
>> Is it possible to get blackfin working with a vanilla gcc release yet, or do
>> you still need out-of-tree patches?  (I have a blackfin board I got at OLS,
>> but it needs a toolchain I can't reproduce.)
>
> there's plenty of usable binaries available
>
>> fact it would be really nice if qemu grew blackfin support because messing
>
> i imagine it would be ... too bad qemu lacks real documentation
>
>>> There is one problem that I can see that may be more serious for
>>> embedded users.
>>> As far as I can tell the initramfs filesystem is not restricted in size.
>>> You can keep writing files until it uses all available memory.
>>
>> Yup.  There have intermittently been patches to make rootfs be tmpfs instead
>> of ramfs, the most recent of which I remember was:
>> http://lkml.org/lkml/2007/7/13/354
>
> it wouldnt matter.  tmpfs on NOMMU doesnt support any of the options
> like MMU.  look at mm/tiny-shmem.c and init/Kconfig.
> -mike
> --

Right, I still got one bug related to tmpfs/ramfs on NOMMU. That is
the fcntl/cease problem.
Some LTP testcases still fail because of this.

-Bryan

Re: initramfs size limitation

[Rob Landley]

On Sunday 01 June 2008 19:25:11 Mike Frysinger wrote:
> On Sun, Jun 1, 2008 at 8:03 PM, Rob Landley wrote:
> > On Saturday 31 May 2008 07:44:14 Phil Wilshire wrote:
> >> I hope this is the right place and the right sort of question.
> >>
> >> I work closely with the Blackfin systems and they have now integrated
> >> the initramfs generation into their system build. The result is great
> >> the root fs is ready to run from the page cache.
> >
> > Is it possible to get blackfin working with a vanilla gcc release yet, or
> > do you still need out-of-tree patches?  (I have a blackfin board I got at
> > OLS, but it needs a toolchain I can't reproduce.)
>
> there's plenty of usable binaries available

I like to build things from source.  (I'm funny that way, I realize this is 
totally out of place when dealing with Linux, but it's what I do.)  I also 
like to use vanilla release packages where possible, and failing that a patch 
against against the release version.  (Again, a personal idiosyncrasy...)

To me, saying "you need this binary-only toolchain to build" is like 
saying "you need this binary-only driver to boot".

> > fact it would be really nice if qemu grew blackfin support because
> > messing
>
> i imagine it would be ... too bad qemu lacks real documentation

Actually there was documentation on the old stuff if you knew where to dig it 
up (a good starting point was Fabrice's old usenix paper and then I had some 
links I'd collected from there), but they just ripped out the old code 
generator in favor of a new one 
(See http://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00011.html ) so 
there's not much point in going there right now.  (Hopefully they'll have a 
1.0 release within our lifetimes.)

I'm guessing that's why your qemu blackfin work on wh0rd.net stalled last 
year?

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: initramfs size limitation

[Mike Frysinger]

On Tue, Jun 3, 2008 at 4:59 PM, Rob Landley wrote:
> On Sunday 01 June 2008 19:25:11 Mike Frysinger wrote:
>> On Sun, Jun 1, 2008 at 8:03 PM, Rob Landley wrote:
>> > fact it would be really nice if qemu grew blackfin support because
>> > messing
>>
>> i imagine it would be ... too bad qemu lacks real documentation
>
> Actually there was documentation on the old stuff if you knew where to dig it
> up (a good starting point was Fabrice's old usenix paper and then I had some
> links I'd collected from there)

yes, but spotiness doesnt help

> but they just ripped out the old code
> generator in favor of a new one
> (See http://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00011.html ) so
> there's not much point in going there right now.  (Hopefully they'll have a
> 1.0 release within our lifetimes.)

i'm aware the tcg stuff exists, but that's about it.  only hope for
new ports atm is to contract a qemu guy.  not that the qemu guys would
mind.  thanks for the link.
-mike

Re: initramfs size limitation

[Rob Landley]

On Tuesday 03 June 2008 16:54:38 Mike Frysinger wrote:
> > but they just ripped out the old code
> > generator in favor of a new one
> > (See http://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00011.html )
> > so there's not much point in going there right now.  (Hopefully they'll
> > have a 1.0 release within our lifetimes.)
>
> i'm aware the tcg stuff exists, but that's about it.  only hope for
> new ports atm is to contract a qemu guy.  not that the qemu guys would
> mind.  thanks for the link.

I have once again allowed myself to get buried in dayjob, but this one only 
lasts until the end of the month and _then_ I plan to get 
http://landley.net/qemu caught up to the present.

I'm the meantime, I'm still trying to extend my FWL scripts to create a cross 
compiler and bootable filesystem for every target qemu supports.  Mips 
regressed in 2.6.25 because they yanked out the specific qemu target 
(http://www.kernel.org/hg/index.cgi/linux-2.6/rev/79936) so I'm migrating the 
sucker to use a malta kernel .config, and then I can do a 0.4.0 release with 
the distcc trick working out of the box.

(QEMU remains a _marvelous_ toy for doing embedded development, to the point 
it's now kind of hard for me to take seriously a platform that doesn't have a 
decent emulator I can run on my laptop.)

> -mike

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.