Q: Cramfs Vs. Ubifs

Q: Cramfs Vs. Ubifs

[Ran Shalit]

Hello,

I read about cramfs and ubifs, but there is still something I don't
understand here, I hope you can help me with that...
1. It seems reasonable that execution from RAM is faster then Nand flash.
    So If I'm using ubifs, should I copy the executables to RAM before execute ?
2. If there is no need for writable filesystem, cramfs seems a good
choice, but some of the utilities write stuff to /var. How should we
handle that with cramfs ?
3. in terms of boot time, both cramfs and ubifs are equal. is it correct ?

Thank you,

Ran

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Mon, 4 Jun 2012, Ran Shalit wrote:

> I read about cramfs and ubifs, but there is still something I don't
> understand here, I hope you can help me with that...
>
> 1. It seems reasonable that execution from RAM is faster then Nand flash.
>    So If I'm using ubifs, should I copy the executables to RAM before 
> execute ?

Most systems can't execute directly from NAND flash, so that is not really 
an option either way. In fact, since NAND flash cannot be mapped to an 
address space, there is no way to directly execute from NAND flash, 
however, one could imagine a system with some form of memory managment 
scheme which buffers the data so that the flash appears to be directly 
addressable. I don't know if there is such a system though; I've never 
heard of one (but that's not saying much).

The 'copy to RAM' is nothing one normally has to worry about; all the 
binaries are on the flash and loaded into RAM when run. Execute in place 
is a special situation that must be explicitly enabled (and I'm very hazy 
on that subject as I've never worked with it).

> 2. If there is no need for writable filesystem, cramfs seems a good
> choice, but some of the utilities write stuff to /var. How should we
> handle that with cramfs ?

You can create an empty /var directory and then mount a tmpfs on it during 
boot.

> 3. in terms of boot time, both cramfs and ubifs are equal. is it correct 
> ?

I don't know in the general case, but I did run some tests on our systems 
a while ago and the boot times for ubifs and cramfs were essentially 
identical. The main differences where that cramfs takes up less space (15 
vs. 22 MB in our case), but for some reason there was more available 
memory runtime, as it seems the Linux system used a lot of buffers with 
cramfs that were not needed with ubifs - the difference was about 9 MB of 
RAM. Didn't have time to investigate the latter fact further though.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Tue, 5 Jun 2012, Ran Shalit wrote:

>  So If I'm using ubifs, should I copy the executables to RAM before execute ?
>
> Most systems can't execute directly from NAND flash, so that is not really an option either way. In fact, since NAND flash cannot be mapped to an address space, there is no way to directly execute from NAND flash, however, one could imagine a system with some form of memory managment scheme which buffers the data so that the flash appears to be directly addressable. I don't know if there is such a system though; I've never heard of one (but that's not saying much).
>
> The 'copy to RAM' is nothing one normally has to worry about; all the binaries are on the flash and loaded into RAM when run. Execute in place is a special situation that must be explicitly enabled (and I'm very hazy on that subject as I've never worked with it).
>
> So it seems that if also in terms of execution ubifs and cramfs are 
> quite similiar (both are executed in RAM), then I see no advatage of 
> using cramfs over ubifs in general (There are also other advantages of 
> ubifs such as bad block skipping, writable filesystem, power up recovery 
> etc.).

I should make it clear that cramfs can not be run directly on NAND flash 
as it has no concept of bad blocks. But neither can ubifs, which requires 
UBI. So both require UBI.

As a matter of fact, come to think of it, I'm not sure how to run cramfs 
on UBI as it requires a block device which UBI doesn't supply. But I have 
tested it in some way, so it's probably just my mind drawing a blank right 
now. :-)

The only real advantage I can see with cramfs is that it does use up less 
space in the flash for the same amount of data. On the other hand, flash 
space is usually not a big concern in NAND flash systems anyway.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Thomas Petazzoni]

Hello,

Le Tue, 5 Jun 2012 11:29:08 +0200 (CEST),
Ricard Wanderlof <ricard.wanderlof@axis.com> a écrit :

> I should make it clear that cramfs can not be run directly on NAND flash 
> as it has no concept of bad blocks. But neither can ubifs, which requires 
> UBI. So both require UBI.
> 
> As a matter of fact, come to think of it, I'm not sure how to run cramfs 
> on UBI as it requires a block device which UBI doesn't supply. But I have 
> tested it in some way, so it's probably just my mind drawing a blank right 
> now. :-)

You can use gluebi+mtdblock on top of a MTD volume to use a read-only
block filesystem on top of MTD. Or, you can use the ubiblk driver,
which isn't mainline for now, but has been posted multiple times last
year by one of my colleagues.

> The only real advantage I can see with cramfs is that it does use up less 
> space in the flash for the same amount of data. On the other hand, flash 
> space is usually not a big concern in NAND flash systems anyway.

cramfs is kind of useless now that we have squashfs that overcomes the
limitations of cramfs (in number of files and filesystem size) and
provides an even higher compression ratio.

So definitely, if you had to use a read-only block filesystem, you
should go with squashfs. But as Ricard pointed, you cannot directly use
either cramfs or squashfs on top of MTD partitions, since those
filesystems do not handle bad blocks.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: Q: Cramfs Vs. Ubifs

[Ran Shalit]

>>
>> 1. It seems reasonable that execution from RAM is faster then Nand flash.
>>   So If I'm using ubifs, should I copy the executables to RAM before
>> execute ?
>
>
> Most systems can't execute directly from NAND flash, so that is not really
> an option either way. In fact, since NAND flash cannot be mapped to an
> address space, there is no way to directly execute from NAND flash, however,
> one could imagine a system with some form of memory managment scheme which
> buffers the data so that the flash appears to be directly addressable. I
> don't know if there is such a system though; I've never heard of one (but
> that's not saying much).
>
> The 'copy to RAM' is nothing one normally has to worry about; all the
> binaries are on the flash and loaded into RAM when run. Execute in place is
> a special situation that must be explicitly enabled (and I'm very hazy on
> that subject as I've never worked with it).
>
>

Hi Ricard,

If I start several executable from ubifs in some start point, Is it
right to assume that all of them are running in RAM, and there is no
access to flash after this start point of execution ?

Thank you very much,
Ran

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Tue, 12 Jun 2012, Ran Shalit wrote:

>> Most systems can't execute directly from NAND flash, so that is not really
>> an option either way. In fact, since NAND flash cannot be mapped to an
>> address space, there is no way to directly execute from NAND flash, however,
>> one could imagine a system with some form of memory managment scheme which
>> buffers the data so that the flash appears to be directly addressable. I
>> don't know if there is such a system though; I've never heard of one (but
>> that's not saying much).
>>
>> The 'copy to RAM' is nothing one normally has to worry about; all the
>> binaries are on the flash and loaded into RAM when run. Execute in place is
>> a special situation that must be explicitly enabled (and I'm very hazy on
>> that subject as I've never worked with it).

> If I start several executable from ubifs in some start point, Is it
> right to assume that all of them are running in RAM, and there is no
> access to flash after this start point of execution ?

As someone else mentioned on the mtd list, Linux is demand paged which 
means that the code is loaded into memory when needed. If there's a chunk 
of code (Linux uses a page size of 4 kbytes so that's the granularity of 
the paging) that is not used until after a while after the start of 
execution, it won't be loaded until it is actually run.

I don't know if there's a way to force it all into memory when execution 
starts; there probably is.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Ran Shalit]

>> If I start several executable from ubifs in some start point, Is it
>> right to assume that all of them are running in RAM, and there is no
>> access to flash after this start point of execution ?
>
>
> As someone else mentioned on the mtd list, Linux is demand paged which means
> that the code is loaded into memory when needed. If there's a chunk of code
> (Linux uses a page size of 4 kbytes so that's the granularity of the paging)
> that is not used until after a while after the start of execution, it won't
> be loaded until it is actually run.
>
> I don't know if there's a way to force it all into memory when execution
> starts; there probably is.
>
In embedded system it is desired to execute only from RAM, therefore
it seems that ubifs is not best choice here.
It seems that in order to run completely from RAM I need to copy all
the files in ubifs potion to RAM and then execute the application in
RAM. Therefore it seems that best choice here would be to use both
root file system in RAM, and also mount ubifs file system in flash,
for easily flashing executables and other files. Is there a simpler
way that I miss here or any wrong assumptions I've made ?

Many Thanks, Ran

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Wed, 13 Jun 2012, Ran Shalit wrote:

>>> If I start several executable from ubifs in some start point, Is it
>>> right to assume that all of them are running in RAM, and there is no
>>> access to flash after this start point of execution ?
>>
>> As someone else mentioned on the mtd list, Linux is demand paged which means
>> that the code is loaded into memory when needed. If there's a chunk of code
>> (Linux uses a page size of 4 kbytes so that's the granularity of the paging)
>> that is not used until after a while after the start of execution, it won't
>> be loaded until it is actually run.
>>
>> I don't know if there's a way to force it all into memory when execution
>> starts; there probably is.
>>
> In embedded system it is desired to execute only from RAM, therefore
> it seems that ubifs is not best choice here.

The demand paging concept simply means that the code is copied to RAM from 
the file system when needed, it is always executed from RAM. But yes, it 
can mean that the application has to wait for data to get copied during 
execution.

> It seems that in order to run completely from RAM I need to copy all
> the files in ubifs potion to RAM and then execute the application in
> RAM.

Even if you copy all files to a RAM-based file system, such as /tmp, they 
still need to be copied from the file system to directly accessible 
memory before execution. While it may be faster than performing the same 
operation from flash, the demand paging functionality is still in 
operation.

The only way to avoid it would be to force loading of all pages for a 
program at its start, I've never had the need so I don't know how though.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Ran Shalit]

> Even if you copy all files to a RAM-based file system, such as /tmp, they
> still need to be copied from the file system to directly accessible memory
> before execution. While it may be faster than performing the same operation
> from flash, the demand paging functionality is still in operation.
>
> The only way to avoid it would be to force loading of all pages for a
> program at its start, I've never had the need so I don't know how though.
>
>
Yes, that what I meant. At the initialization stage we will copy all
files from ubifs section into root file system in RAM and so I assume
that there will be no any access to flash during application life for
read or write (any desired access will be controlled by application).
I wonder why I didn't find any information about disadvantage of using
ubifs as root file system in embedded system in terms of access to
flash during application life.

Regards, Ran

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Wed, 13 Jun 2012, Ran Shalit wrote:

>> Even if you copy all files to a RAM-based file system, such as /tmp, they
>> still need to be copied from the file system to directly accessible memory
>> before execution. While it may be faster than performing the same operation
>> from flash, the demand paging functionality is still in operation.
>>
>> The only way to avoid it would be to force loading of all pages for a
>> program at its start, I've never had the need so I don't know how though.
>>
>>
> Yes, that what I meant. At the initialization stage we will copy all
> files from ubifs section into root file system in RAM and so I assume
> that there will be no any access to flash during application life for
> read or write (any desired access will be controlled by application).
> I wonder why I didn't find any information about disadvantage of using
> ubifs as root file system in embedded system in terms of access to
> flash during application life.

I think because it normally is not a problem. Also, the same goes for any 
file system which resides on the flash, it's not a ubifs-specific problem.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Nathan Lynch]

On Wed, 2012-06-13 at 18:25 +0300, Ran Shalit wrote:
> > Even if you copy all files to a RAM-based file system, such as /tmp, they
> > still need to be copied from the file system to directly accessible memory
> > before execution. While it may be faster than performing the same operation
> > from flash, the demand paging functionality is still in operation.
> >
> > The only way to avoid it would be to force loading of all pages for a
> > program at its start, I've never had the need so I don't know how though.
> >
> >
> Yes, that what I meant. At the initialization stage we will copy all
> files from ubifs section into root file system in RAM and so I assume
> that there will be no any access to flash during application life for
> read or write (any desired access will be controlled by application).
> I wonder why I didn't find any information about disadvantage of using
> ubifs as root file system in embedded system in terms of access to
> flash during application life.

This isn't an issue specific to UBIFS.  Look into mlock/mlockall if your
application needs to avoid latencies associated with demand-paging from
flash or any other storage.

Re: Q: Cramfs Vs. Ubifs

[Ran Shalit]

>> I wonder why I didn't find any information about disadvantage of using
>> ubifs as root file system in embedded system in terms of access to
>> flash during application life.
>>
>I think because it normally is not a problem. Also, the same goes for any file system which resides on the flash, it's not a ubifs-specific problem.
>
> This isn't an issue specific to UBIFS.  Look into mlock/mlockall if your
> application needs to avoid latencies associated with demand-paging from
> flash or any other storage.
>
Can you please elaborate why accessing the flash while main "real-time
application" is running it is not a problem. I also read in
free-electrons tutorial on embedded linux such recommendation for
using file system such as cramfs, ubifs, which also access the flash
during execution.
Why is it not a problem for embedded linux ? From my experience with
other embedded OS, it is not recommended to access flash during
execution.
Many thanks, Ran

Re: Q: Cramfs Vs. Ubifs

[Nathan Lynch]

On Thu, 2012-06-14 at 08:37 +0300, Ran Shalit wrote:
> >> I wonder why I didn't find any information about disadvantage of using
> >> ubifs as root file system in embedded system in terms of access to
> >> flash during application life.
> >>
> >
> > This isn't an issue specific to UBIFS.  Look into mlock/mlockall if your
> > application needs to avoid latencies associated with demand-paging from
> > flash or any other storage.
> >
> Can you please elaborate why accessing the flash while main "real-time
> application" is running it is not a problem. I also read in
> free-electrons tutorial on embedded linux such recommendation for
> using file system such as cramfs, ubifs, which also access the flash
> during execution.
> Why is it not a problem for embedded linux ? From my experience with
> other embedded OS, it is not recommended to access flash during
> execution.

Did you read the man page for mlockall?  It does seem to address your
concern as I understand it.

(This has become off-topic for the MTD list IMO.)

Re: Q: Cramfs Vs. Ubifs

[Ricard Wanderlof]

On Thu, 14 Jun 2012, Nathan Lynch wrote:

>
>> Can you please elaborate why accessing the flash while main "real-time
>> application" is running it is not a problem. I also read in
>> free-electrons tutorial on embedded linux such recommendation for
>> using file system such as cramfs, ubifs, which also access the flash
>> during execution.
>> Why is it not a problem for embedded linux ? From my experience with
>> other embedded OS, it is not recommended to access flash during
>> execution.

I think the question to ask is 'why is it a problem'? If you can meet your 
real time requirements even if there is latency due to parts of the 
application being fetched (i.e. paged in) runtime, it is not a problem.

If you do rely on your application having millisecond response times, then 
you have quite stringent real time requirements, and you may need to lock 
your application in memory to avoid latency due to paging. You may also 
have to look at configuring the Linux kernel so that it doesn't introduce 
any latency due to task switching. And there may be other issues in that 
case, with paging from mass storage being but one of them.

For reference, the company I work for makes network cameras, running 
embedded Linux, which is definitely a real time application (streaming 
live video), but our requirements are such that we have not needed to take 
measures such as mlockall in order to keep applications in memory at all 
times.

So I wouldn't say as a general rule that accessing flash memory during 
execution is a problem, it all depends on what your requirements are.

> On Thu, 2012-06-14 at 08:37 +0300, Ran Shalit wrote:
>
> Did you read the man page for mlockall?  It does seem to address your
> concern as I understand it.
>
> (This has become off-topic for the MTD list IMO.)

Principally, yes, but since MTD devices tend to be used primarily in 
embedded systems, and they are often the ones with the most real time 
intensive requirements, there is a certain relevancy even if it is not 
directly related to mtd.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Q: Cramfs Vs. Ubifs

[Thomas Petazzoni]

Hello,

Ricard has already answered your points, but I felt that adding an
additional detail would be useful. See below.

Le Mon, 4 Jun 2012 21:17:34 +0200,
Ran Shalit <ranshalit@gmail.com> a écrit :

> I read about cramfs and ubifs, but there is still something I don't
> understand here, I hope you can help me with that...
> 1. It seems reasonable that execution from RAM is faster then Nand flash.
>     So If I'm using ubifs, should I copy the executables to RAM before execute ?

This is not needed. The Linux kernel uses the MMU to do demand paging
so load on demand into RAM the portions of the program that are
actually being used. So Linux userspace applications are always
executed from RAM (unless, as Ricard pointed out, you use some XIP
mechanism, but that requires NOR flash and special filesystems, so it's
a very uncommon situation these days). However, an application is not
completely loaded at once at the beginning of its execution, parts of
it are loaded on demand during the execution.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com