ramdisk/tmpfs/ramfs/memfs ?

ramdisk/tmpfs/ramfs/memfs ?

[Padraig Brady]

Hi,

I'm working on an embedded system here which has no harddisk.
So, I can't swap to disk and need to have /var & /tmp in RAM.
I'm confused between the various options for in RAM file-
systems. At the moment I've created a ramdisk and made an 
ext2 partition in it (which is compressed as I applied the 
e2compr patch), which is working fine. Anyway questions:

1. I presume the kernel is clever enough to not cache any
   files from these filesystems? Would it ever need to?
2. Is tmpfs is basically swap and /tmp together in a ramdisk?
   The advantage being you need to reserve less RAM for both
   together than seperately?
3. If I've no backing store (harddisk?) is there any advantage 
   of using tmpfs instead of ramfs? Also does tmpfs need a 
   backing store?
5. Can you set size limits on ramfs/tmpfs/memfs?
6. Is a ramdisk resizable like the others. If so, do you have
   to delete/recreate or umount/resize a fs (e.g. ext2) every
   time it's resized? Do ramfs/tmpfs/memfs do this transparently?
   Are ramdisks resizable in kernel 2.2?
7. What's memfs?
8. Is there a way I can get transparent compression like I now
   have using a ramdisk+ext2+e2compr with ramfs et al?
9. Apart from this transparent compression, is there any other
   functionality ext2 would have over ramfs for e.g, for /tmp
   & /var? Also would ramfs have less/more speed over ext2?

thanks,
Padraig.

Re: ramdisk/tmpfs/ramfs/memfs ?

[Bjorn Wesen]

On Thu, 26 Apr 2001, Padraig Brady wrote:
> I'm working on an embedded system here which has no harddisk.
> So, I can't swap to disk and need to have /var & /tmp in RAM.
> I'm confused between the various options for in RAM file-
> systems. At the moment I've created a ramdisk and made an 
> ext2 partition in it (which is compressed as I applied the 
> e2compr patch), which is working fine. Anyway questions:

Ouch.. yes you had to do stuff like that in the old days but it's very 
cumbersome and inefficient compared to ramfs for what you're trying to do.

> 1. I presume the kernel is clever enough to not cache any
>    files from these filesystems? Would it ever need to?

You always need to "cache" pages read. Because a page is the smallest
possible granularity for the MMU, and a block-based filesystem does not
need to be page-aligned, so it's impossible to do it otherwise in a
general way.

> 3. If I've no backing store (harddisk?) is there any advantage 
>    of using tmpfs instead of ramfs? Also does tmpfs need a 
>    backing store?

I don't know what tmpfs does actually, but if it is like you suggest (a
ramfs that can be swapped out ?) then you don't need it obviously (since
you don't have any swap).

ramfs simply inserts any files written into the kernels cache and tells it
not to forget it. it can't get much more simple than that.

> 5. Can you set size limits on ramfs/tmpfs/memfs?

i don't think you can set a limit in the current ramfs implementation but
it would not be particularly difficult to make it work I think

> 6. Is a ramdisk resizable like the others. If so, do you have
>    to delete/recreate or umount/resize a fs (e.g. ext2) every
>    time it's resized? Do ramfs/tmpfs/memfs do this transparently?
>    Are ramdisks resizable in kernel 2.2?

ramfs does not need any "resizing" because there is no filesystem behind
it. there is only the actual file data and metadata in the cache itself.
if you delete a file, it disapperas, if you create a new one new pages are
brought in.

> 7. What's memfs?
> 8. Is there a way I can get transparent compression like I now
>    have using a ramdisk+ext2+e2compr with ramfs et al?

you could try using jffs2 on a RAM-simulated MTD partition. i think that
would work but i have not tried it..

> 9. Apart from this transparent compression, is there any other
>    functionality ext2 would have over ramfs for e.g, for /tmp
>    & /var? Also would ramfs have less/more speed over ext2?

ramfs has all the bells and whistles you need except size limiting. and
obviously its faster than simulating a harddisk in ram and using ext2 on
it.. 

-bw

Re: ramdisk/tmpfs/ramfs/memfs ?

[Marko Kreen]

On Thu, Apr 26, 2001 at 08:48:26PM +0200, Bjorn Wesen wrote:
> On Thu, 26 Apr 2001, Padraig Brady wrote:
> > 3. If I've no backing store (harddisk?) is there any advantage 
> >    of using tmpfs instead of ramfs? Also does tmpfs need a 
> >    backing store?
> 
> I don't know what tmpfs does actually, but if it is like you suggest (a
> ramfs that can be swapped out ?) then you don't need it obviously (since
> you don't have any swap).
> 
> ramfs simply inserts any files written into the kernels cache and tells it
> not to forget it. it can't get much more simple than that.
> 
> > 5. Can you set size limits on ramfs/tmpfs/memfs?
> 
> i don't think you can set a limit in the current ramfs implementation but
> it would not be particularly difficult to make it work I think

tmpfs is basically ramfs with limits.

-- 
marko

Re: ramdisk/tmpfs/ramfs/memfs ?

[H. Peter Anvin]

Followup to:  <20010426223950.C3683@l-t.ee>
By author:    Marko Kreen <marko@l-t.ee>
In newsgroup: linux.dev.kernel
> 
> tmpfs is basically ramfs with limits.
> 

... and swappable.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt

Re: ramdisk/tmpfs/ramfs/memfs ?

[mirabilos]

> > tmpfs is basically ramfs with limits.
> > 
> 
> ... and swappable.
> 
> -hpa

Hmmm and what's shmfs? Precedessor of tmpfs?
I even cant remember which one I use for /tmp ;-)

-mirabilos

Re: ramdisk/tmpfs/ramfs/memfs ?

[Christoph Rohland]

On Fri, 27 Apr 2001, eccesys@topmail.de wrote:
>> > tmpfs is basically ramfs with limits.
>> > 
>> 
>> ... and swappable.
>> 
>> -hpa
> 
> Hmmm and what's shmfs? Precedessor of tmpfs?

Yes.

> I even cant remember which one I use for /tmp ;-)

You can mount tmpfs also with type "shm" for compatibility. Type "shm"
will be marked as obsolete in 2.5

Greetings
		Christoph

Re: ramdisk/tmpfs/ramfs/memfs ?

[H. Peter Anvin]

Followup to:  <Pine.LNX.3.96.1010426203656.22847A-100000@medusa.sparta.lu.se>
By author:    Bjorn Wesen <bjorn@sparta.lu.se>
In newsgroup: linux.dev.kernel
> 
> > 5. Can you set size limits on ramfs/tmpfs/memfs?
> 
> i don't think you can set a limit in the current ramfs implementation but
> it would not be particularly difficult to make it work I think
> 

It's a little more painful than you'd think for the simple reason that
ramfs currently contains no space accounting whatsoever, which
probably is a bad thing.  It definitely gave me some serious pause
when I was working on SuperRescue 1.3, since I had no way of
reasonably judging how big my ramfs actually was.  The only way I
could get a reasonable idea was rebooting with various mem=
parameters.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt

Re: ramdisk/tmpfs/ramfs/memfs ?

[Ingo Oeser]

On Thu, Apr 26, 2001 at 01:49:05PM -0700, H. Peter Anvin wrote:
> > > 5. Can you set size limits on ramfs/tmpfs/memfs?

Yes, there is a patch for this.

> > i don't think you can set a limit in the current ramfs implementation but
> > it would not be particularly difficult to make it work I think
> It's a little more painful than you'd think for the simple reason that
> ramfs currently contains no space accounting whatsoever, which
> probably is a bad thing.  It definitely gave me some serious pause
> when I was working on SuperRescue 1.3, since I had no way of
> reasonably judging how big my ramfs actually was.  The only way I
> could get a reasonable idea was rebooting with various mem=
> parameters.

The patched variant gives to all of it. Even several kinds of
limits (inodes, dentries, ram pages).

I use this patch in production. 

The ac-Kernels all have this patch included, which are sometimes
more stable anyway these day. ;-)

HTH

Regards

Ingo Oeser
-- 
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
         <<<<<<<<<<<<     been there and had much fun   >>>>>>>>>>>>

Re: ramdisk/tmpfs/ramfs/memfs ?

[David Woodhouse]

bjorn@sparta.lu.se said:
>  you could try using jffs2 on a RAM-simulated MTD partition. i think
> that would work but i have not tried it..

It works. Most of the early testing and development was done on it. It 
wouldn't give you dynamic sizing like ramfs though. 

It would be nice to have a version of ramfs which compresses pages into a 
separate backing store when they're unused. Shame somebody nicked the name 
'cramfs' for something else, really :)

But I'm confused. Padraig, if you have no backing store, where do the 
initial contents of your root filesystem come from?

--
dwmw2

Re: ramdisk/tmpfs/ramfs/memfs ?

[mirabilos]

> >  you could try using jffs2 on a RAM-simulated MTD partition. i think
> > that would work but i have not tried it..
>
> It works. Most of the early testing and development was done on it. It
> wouldn't give you dynamic sizing like ramfs though.
>
> It would be nice to have a version of ramfs which compresses pages
into a
> separate backing store when they're unused. Shame somebody nicked the
name

great... especially for my boot/rootdisks on an 8/16MB system which
start
swapon right in the /linuxrc

> 'cramfs' for something else, really :)

This should be names cromfs as IIRC it isn't writable.

>
> But I'm confused. Padraig, if you have no backing store, where do the
> initial contents of your root filesystem come from?

Netboot? Floppies? (as for me)

-mirabilos

Re: ramdisk/tmpfs/ramfs/memfs ?

[Padraig Brady]

David Woodhouse wrote:

>bjorn@sparta.lu.se said:
>
>> you could try using jffs2 on a RAM-simulated MTD partition. i think
>>that would work but i have not tried it..
>>
>
>It works. Most of the early testing and development was done on it. It 
>wouldn't give you dynamic sizing like ramfs though. 
>
>It would be nice to have a version of ramfs which compresses pages into a 
>separate backing store when they're unused. Shame somebody nicked the name 
>'cramfs' for something else, really :)
>

Hmm must look into getting ramfs working with
http://linuxcompressed.sourceforge.net/
seems like the best of both worlds.

Padraig

Re: ramdisk/tmpfs/ramfs/memfs ?

[David Woodhouse]

padraig@antefacto.com said:
> btw I get my initial root filesystem from a compact flash that can be
> accessed just like a hardisk. It's writeable also like a harddisk, but
> we boot with it readonly, and only mount it rw if we want to save
> config or whatever. We definitely wouldn't swap to it as it has
> limited erase/write cycles. The filesystem is compressed ext2.

Why copy it into RAM? Why not use cramfs and either turn the writable
directories into symlinks into a ramfs which you create at boot time, or 
union-mount a ramfs over the top of it?


padraig@antefacto.com said:
> As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk is not
> an option as the only advantage would be journalling, you still can't
> resize. IMHO JFFS is only required where you have flash without an IDE
> interface.

True. We are currently lacking a compact, compressing, journalling 
filesystem for use on block devices. It's been suggested that we could make 
JFFS2 work on them, by making a fake MTD device which uses a block device 
as backing store. Nobody's yet shown me the code though :)

--
dwmw2

Re: ramdisk/tmpfs/ramfs/memfs ?

[mirabilos]

> > btw I get my initial root filesystem from a compact flash that can
be
> > accessed just like a hardisk. It's writeable also like a harddisk,
but
> > we boot with it readonly, and only mount it rw if we want to save
> > config or whatever. We definitely wouldn't swap to it as it has
> > limited erase/write cycles. The filesystem is compressed ext2.
>
> Why copy it into RAM? Why not use cramfs and either turn the writable
> directories into symlinks into a ramfs which you create at boot time,
or
> union-mount a ramfs over the top of it?

Hey! SOunds great! How to do it?
I tried to # mount --bind the directories but they do not show
me both entries. I have romfs root and tmpfs.

-mirabilos

Re: ramdisk/tmpfs/ramfs/memfs ?

[David L. Parsley]

David Woodhouse wrote:

> Why copy it into RAM? Why not use cramfs and either turn the writable
> directories into symlinks into a ramfs which you create at boot time, or
> union-mount a ramfs over the top of it?
  ^^^^^^^^^^^

I didn't think we had union-mounting support... does it exist and
I've somehow missed it?

regards,
	David

-- 
David L. Parsley
Network Administrator
Roanoke College

Re: ramdisk/tmpfs/ramfs/memfs ?

[Padraig Brady]

David Woodhouse wrote:
> 
> padraig@antefacto.com said:
> > btw I get my initial root filesystem from a compact flash that can be
> > accessed just like a hardisk. It's writeable also like a harddisk, but
> > we boot with it readonly, and only mount it rw if we want to save
> > config or whatever. We definitely wouldn't swap to it as it has
> > limited erase/write cycles. The filesystem is compressed ext2.
> 
> Why copy it into RAM? Why not use cramfs and either turn the writable
> directories into symlinks into a ramfs which you create at boot time, or
> union-mount a ramfs over the top of it?

? I never said I copied it into RAM. The ramfs is just used for /tmp &
/var
which are symlinks on the flash. I didn't know you could read/write
cramfs
like ext2/JFFS2? I still want to be able to update/create/delete files
like a normal ext2 partition. Oh I see the confusion, sorry my fault,
when
I said the root filesystem is compressed ext2, I meant it's ext2 with
chattr +c done on all files (the e2compr patch implmenents it). By the
way why isn't e2compr a standard part of the kernel. I've have no
problems
at all with it.

I didn't know about union-mounting, seems useful.

As for whether to use tmpfs/ramfs I think they do basically the same
thing 
only the implementation is different. tmpfs uses shared mem and so is 
probably more portable than ramfs which it tightly coupled with VFS. I
think I'll use ramfs (with limits patch) so.

> padraig@antefacto.com said:
> > As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk is not
> > an option as the only advantage would be journalling, you still can't
> > resize. IMHO JFFS is only required where you have flash without an IDE
> > interface.
> 
> True. We are currently lacking a compact, compressing, journalling
> filesystem for use on block devices. It's been suggested that we could make
> JFFS2 work on them, by making a fake MTD device which uses a block device
> as backing store. Nobody's yet shown me the code though :)

How much more efficent is JFFS than say ext3+e3compr, wrt:
logic size/logic speed/RAM requirements/filesystem structure size.

> --
> dwmw2

Padraig.

Re: ramdisk/tmpfs/ramfs/memfs ?

[Xavier Bestel]

Le 27 Apr 2001 18:53:07 +0100, Padraig Brady a écrit :

> How much more efficent is JFFS than say ext3+e3compr, wrt:
> logic size/logic speed/RAM requirements/filesystem structure size.

JFFS doesn't have to use the FTL layer between its block device and the
flash - that's a lot already !

Xav

Re: ramdisk/tmpfs/ramfs/memfs ?

[David Woodhouse]

parsley@linuxjedi.org said:
>  I didn't think we had union-mounting support... does it exist and
> I've somehow missed it? 

I think you need Al Viro's namespace stuff for it to work properly.

--
dwmw2

Re: ramdisk/tmpfs/ramfs/memfs ?

[Padraig Brady]

Thanks for all the replies!

Would anyone have any objections to me writing a 
Documentation/ramdisks.txt or Documentation/filesystems/ram.txt
as this stuff is very hard to find info on?

Anyway seems like ramfs is the way to go with the limits patch
applied. I loose compression, but gain as there is no data
duplication in RAM, and also I don't need to preallocate space
for a partition. If I understand correctly ramfs just points
to the file data which are pages in the cache marked not to be
uncached. Doh! is ramfs supported in 2.2?

dwmw2 mentioned, for ramfs, it would be useful to compress unused
pages (to someplace in RAM would be very good for my application),
but I guess this would add a lot of complexity.

btw I get my initial root filesystem from a compact flash that
can be accessed just like a hardisk. It's writeable also like
a harddisk, but we boot with it readonly, and only mount it rw
if we want to save config or whatever. We definitely wouldn't
swap to it as it has limited erase/write cycles. The filesystem
is compressed ext2.

I don't have swap so don't need tmpfs, but could probably
use it anyway without a backing store? Anyway why was ramfs
created if tmpfs existed, unless tmpfs requires backing store?
They both seem to have been written around the same time?

As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk
is not an option as the only advantage would be journalling, you
still can't resize. IMHO JFFS is only required where you have
flash without an IDE interface.

cheers,
Padraig.


Bjorn Wesen wrote:
> 
> On Thu, 26 Apr 2001, Padraig Brady wrote:
> > I'm working on an embedded system here which has no harddisk.
> > So, I can't swap to disk and need to have /var & /tmp in RAM.
> > I'm confused between the various options for in RAM file-
> > systems. At the moment I've created a ramdisk and made an
> > ext2 partition in it (which is compressed as I applied the
> > e2compr patch), which is working fine. Anyway questions:
> 
> Ouch.. yes you had to do stuff like that in the old days but it's very
> cumbersome and inefficient compared to ramfs for what you're trying to do.
> 
> > 1. I presume the kernel is clever enough to not cache any
> >    files from these filesystems? Would it ever need to?
> 
> You always need to "cache" pages read. Because a page is the smallest
> possible granularity for the MMU, and a block-based filesystem does not
> need to be page-aligned, so it's impossible to do it otherwise in a
> general way.
> 
> > 3. If I've no backing store (harddisk?) is there any advantage
> >    of using tmpfs instead of ramfs? Also does tmpfs need a
> >    backing store?
> 
> I don't know what tmpfs does actually, but if it is like you suggest (a
> ramfs that can be swapped out ?) then you don't need it obviously (since
> you don't have any swap).
> 
> ramfs simply inserts any files written into the kernels cache and tells it
> not to forget it. it can't get much more simple than that.
> 
> > 5. Can you set size limits on ramfs/tmpfs/memfs?
> 
> i don't think you can set a limit in the current ramfs implementation but
> it would not be particularly difficult to make it work I think
> 
> > 6. Is a ramdisk resizable like the others. If so, do you have
> >    to delete/recreate or umount/resize a fs (e.g. ext2) every
> >    time it's resized? Do ramfs/tmpfs/memfs do this transparently?
> >    Are ramdisks resizable in kernel 2.2?
> 
> ramfs does not need any "resizing" because there is no filesystem behind
> it. there is only the actual file data and metadata in the cache itself.
> if you delete a file, it disapperas, if you create a new one new pages are
> brought in.
> 
> > 7. What's memfs?
> > 8. Is there a way I can get transparent compression like I now
> >    have using a ramdisk+ext2+e2compr with ramfs et al?
> 
> you could try using jffs2 on a RAM-simulated MTD partition. i think that
> would work but i have not tried it..
> 
> > 9. Apart from this transparent compression, is there any other
> >    functionality ext2 would have over ramfs for e.g, for /tmp
> >    & /var? Also would ramfs have less/more speed over ext2?
> 
> ramfs has all the bells and whistles you need except size limiting. and
> obviously its faster than simulating a harddisk in ram and using ext2 on
> it..
> 
> -bw
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: ramdisk/tmpfs/ramfs/memfs ?

[Bjorn Wesen]

On Fri, 27 Apr 2001, Padraig Brady wrote:
> for a partition. If I understand correctly ramfs just points
> to the file data which are pages in the cache marked not to be

It does not even do that - as of 2.4, the VFS in the kernel also knows how
to cache a filestructure itself. It's in the dentry-cache. So ramfs just
provides the thin mapping between VFS operations and the VFS caches
(dentries, inodes, pages) like any other 2.4 filesystem - with the
difference that ramfs does not need to know anything about actually
transferring the cache entries to a backing store (a physical filesystem).

Take a look at fs/ramfs/inode.c, it's just some hundred odd lines of
code and worth reading to find out more about how 2.4's VFS works.

> uncached. Doh! is ramfs supported in 2.2?

Don't think so, for the above reason.

-BW

Re: ramdisk/tmpfs/ramfs/memfs ?

[Christoph Rohland]

Hi Padraig,

On Fri, 27 Apr 2001, Padraig Brady wrote:
> I don't have swap so don't need tmpfs, but could probably
> use it anyway without a backing store? 

Yes, it does not need backing store.

> Anyway why was ramfs created if tmpfs existed, unless tmpfs requires
> backing store?  They both seem to have been written around the same
> time?

- shm fs was written as a specialized fs to implement POSIX shared
  memory based on SYSV shm.
- ramfs was introduced shortly after shm fs and was meant as a
  programming example for a minimal virtual filesystem. 
- Later shm fs was redone to use the same methods like ramfs but still
  was only useable for shared memory.
- After the release of 2.4.0, I extended shm fs to support read/write
  and thus be tmpfs and since then it can replace ramfs.

Greetings
		Christoph

Re: ramdisk/tmpfs/ramfs/memfs ?

[Christoph Rohland]

Hi Padraig,

On Thu, 26 Apr 2001, Padraig Brady wrote:
> 2. Is tmpfs is basically swap and /tmp together in a ramdisk?
>    The advantage being you need to reserve less RAM for both
>    together than seperately?

tmpfs is ramfs+swap+limits. It is not using ramdisks and is not
related to them.

> 3. If I've no backing store (harddisk?) is there any advantage 
>    of using tmpfs instead of ramfs? Also does tmpfs need a 
>    backing store?

Probably yes, since you spare a little bit kernel memory. most of
tmpfs is unconditionally in the kernel for shared mappings. So the
actual CONFIG_TMPFS only adds some small functions to the kernel to
export this to usre space.

Greetings
		Christoph