What File System supports Application XIP

What File System supports Application XIP

[colin]

Hi there,
We are developing embedded Linux system. Performance is our consideration.
We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in
RAM, and run in XIP (eXecute In Place)  manners.
I know that Cramfs has supported Application XIP. Is there any other FS that
also supports it? Ramdisk? Ramfs? Romfs?

Thanks and regards,
Colin

Re: What File System supports Application XIP

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

On Thu, 2004-09-09 at 10:55, colin wrote:
> Hi there,
> We are developing embedded Linux system. Performance is our consideration.
> We hope some applications can run as fast as possible,

well ramfs by definition is XIP :)

but I guess the filesystem comes from flash somewhere at which point
jffs2 with compression might be a better choice; if you have enough ram
then the apps run from the pagecache anyway and compression keeps you
from transfering too much data from the slower flash. It's not XIP but I
don't think you really want XIP...

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Re: What File System supports Application XIP

[colin]

Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is
executed,
"exec", which is called by sh, should first copy every section of the
application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?

Thanks and regards,
Colin

----- Original Message ----- 
From: "Arjan van de Ven" <arjanv@redhat.com>
To: "colin" <colin@realtek.com.tw>
Cc: <linux-kernel@vger.kernel.org>
Sent: Thursday, September 09, 2004 5:18 PM
Subject: Re: What File System supports Application XIP

> On Thu, 2004-09-09 at 10:55, colin wrote:
> > Hi there,
> > We are developing embedded Linux system. Performance is our
consideration.
> > We hope some applications can run as fast as possible,

> well ramfs by definition is XIP :)

> but I guess the filesystem comes from flash somewhere at which point
> jffs2 with compression might be a better choice; if you have enough ram
> then the apps run from the pagecache anyway and compression keeps you
> from transfering too much data from the slower flash. It's not XIP but I
> don't think you really want XIP...

Re: Re: What File System supports Application XIP

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 465 bytes --]

On Thu, 2004-09-09 at 11:45, colin wrote:
> Hi,
> How does ramfs offer application XIP ability?
> I mean, when the ramfs image is mounted and the application in it is
> executed,
> "exec", which is called by sh, should first copy every section of the
> application to RAM and then jump to the text section.
> How do I avoid the stage copying text section to RAM?

this is not how linux works. programs execute directly from the
pagecache without copy.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: What File System supports Application XIP

[Tim Bird]

Arjan van de Ven wrote:
> On Thu, 2004-09-09 at 11:45, colin wrote:
>> Hi,
>> How does ramfs offer application XIP ability?
>> I mean, when the ramfs image is mounted and the application in it is
>> executed,
>> "exec", which is called by sh, should first copy every section of the
>> application to RAM and then jump to the text section.
>> How do I avoid the stage copying text section to RAM?
> 
> this is not how linux works. programs execute directly from the
> pagecache without copy.

Most other filesystems populate the pagecache with I/O, presumably.
In the case of a ramfs, is the page mapped directly from the fs
into the pagecache without a copy?

=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: tim.bird@am.sony.com
=============================

Re: What File System supports Application XIP

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 234 bytes --]


> Most other filesystems populate the pagecache with I/O, presumably.
> In the case of a ramfs, is the page mapped directly from the fs
> into the pagecache without a copy?

actually ramfs exclusively lives in the pagecache.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: What File System supports Application XIP

[Robert Love]

On Thu, 2004-09-09 at 09:56 -0700, Tim Bird wrote:

> Most other filesystems populate the pagecache with I/O, presumably.
> In the case of a ramfs, is the page mapped directly from the fs
> into the pagecache without a copy?

ramfs _is_ pagecache.

it is cool like that.

	Robert Love

Re: Re: What File System supports Application XIP

[colin]

Hi Robert,
Sorry. I donot quite understand. My English is poor... :-(
Do you mean there wonot be another duplicated text section in RAM when
application in Ramfs is executed?

Regards,
Colin


----- Original Message ----- 
From: "Robert Love" <rml@ximian.com>
To: "Tim Bird" <tim.bird@am.sony.com>
Cc: <arjanv@redhat.com>; "colin" <colin@realtek.com.tw>;
<linux-kernel@vger.kernel.org>
Sent: Friday, September 10, 2004 1:38 AM
Subject: 　[*垃圾郵件*] Re: What File System supports Application XIP


> On Thu, 2004-09-09 at 09:56 -0700, Tim Bird wrote:
>
> > Most other filesystems populate the pagecache with I/O, presumably.
> > In the case of a ramfs, is the page mapped directly from the fs
> > into the pagecache without a copy?
>
> ramfs _is_ pagecache.
>
> it is cool like that.
>
> Robert Love
>
>
>

Re: Re: What File System supports Application XIP

[Robert Love]

On Fri, 2004-09-10 at 10:12 +0800, colin wrote:

> Sorry. I donot quite understand. My English is poor... :-(
> Do you mean there wonot be another duplicated text section in RAM when
> application in Ramfs is executed?

This is correct, Colin.

Ramfs is essentially an in-memory filesystem implemented in the page
cache.

	Robert Love

Re: What File System supports Application XIP

[Paulo Marques]

colin wrote:
> 
> Hi there,
> We are developing embedded Linux system. Performance is our consideration.
> We hope some applications can run as fast as possible,
> and are think if they can be put in a filesystem image, which resides in
> RAM, and run in XIP (eXecute In Place)  manners.
> I know that Cramfs has supported Application XIP. Is there any other FS that
> also supports it? Ramdisk? Ramfs? Romfs?

Obvisously cramfs can not support XIP, because the "in-place" image
is compressed (unless you have a processor that can execute compressed
code :)

AFAIK only tmpfs supports XIP because it works on a higher level
without using block devices underneath. Ramdisks are simply RAM
block devices that behave like any other block device.

You can have a compressed image in flash (for instance), decompress
everything into a tmpfs and execute from there.

I'm not sure, however, that this will be such a performance gain.

If you use cramfs (for instance) then the kernel will uncompress
and run only the pages that are needed, and they will be cached in
page cache so that they will be available again when needed. This
way you only waste the RAM you actually need, and can still drop
old pages if the application needs more RAM.

Just my two cents,

-- 
Paulo Marques - www.grupopie.com

To err is human, but to really foul things up requires a computer.
Farmers' Almanac, 1978

Re: What File System supports Application XIP

[David Woodhouse]

On Thu, 2004-09-09 at 10:19 +0100, Paulo Marques wrote:
> colin wrote:
> > 
> > Hi there,
> > We are developing embedded Linux system. Performance is our consideration.
> > We hope some applications can run as fast as possible,
> > and are think if they can be put in a filesystem image, which resides in
> > RAM, and run in XIP (eXecute In Place)  manners.
> > I know that Cramfs has supported Application XIP. Is there any other FS that
> > also supports it? Ramdisk? Ramfs? Romfs?
> 
> Obvisously cramfs can not support XIP, because the "in-place" image
> is compressed (unless you have a processor that can execute compressed
> code :)

Actually there are hacks floating around which do let you use XIP with
cramfs -- obviously you have to dispense with compression for the files
you want to access that way.

You won't gain at runtime by using XIP though. Your code and data will
end up in RAM _whatever_ file system you use, and you'll be running from
page cache.

You may get a _slightly_ faster startup time after reboot if you use XIP
from flash, because it doesn't have to be loaded into RAM first. But
that comes at the cost of making it all a low slower during normal
operation -- it's a lot slower to fetch icache lines from flash than it
is from RAM.

Arjan's right -- you almost certainly don't really want XIP. 

-- 
dwmw2

Re: What File System supports Application XIP

[Tim Bird]

David Woodhouse wrote:
> On Thu, 2004-09-09 at 10:19 +0100, Paulo Marques wrote:
>> colin wrote:
>> > 
>> > Hi there,
>> > We are developing embedded Linux system. Performance is our consideration.
>> > We hope some applications can run as fast as possible,
>> > and are think if they can be put in a filesystem image, which resides in
>> > RAM, and run in XIP (eXecute In Place)  manners.
>> > I know that Cramfs has supported Application XIP. Is there any other FS that
>> > also supports it? Ramdisk? Ramfs? Romfs?
>> 
>> Obvisously cramfs can not support XIP, because the "in-place" image
>> is compressed (unless you have a processor that can execute compressed
>> code :)
> 
> Actually there are hacks floating around which do let you use XIP with
> cramfs -- obviously you have to dispense with compression for the files
> you want to access that way.

The patches I've seen require setting the CRAMFS_LINEAR option, to turn on
linear addressing for cramfs, and CRAMFS_LINEAR_XIP.  The result of these
is to dispense with compression.

> 
> You won't gain at runtime by using XIP though. Your code and data will
> end up in RAM _whatever_ file system you use, and you'll be running from
> page cache.

Really?  I thought that when you XIPed from flash, the page was mapped
directly back to the flash memory address.  There shouldn't be a RAM
copy of the page at all.

Greg Ungerer (uClinux maintainer) once wrote:
> [Application XIP provides a] win of keeping the application code in
> flash even when that is shared. Can make a difference on small memory
> systems.
>  
> It also helps alleviate the contiguous memory problem (or memory 
> fragmentation if you prefer) when you don't have an MMU. We need to 
> be able to allocate a big enough contigous memory region to load 
> the text into. Can be a problem on systems that have been running 
> for a while and free memory is fragmented. If the application can 
> be run XIP from flash then you at least don't need to worry about 
> that. (This is a very real problem on small RAM systems). 

In the CE Linux Forum, we've been investigating Application XIP for
two purposes: reduction in RAM footprint, and faster startup time.
However, those both come at the sacrifice of runtime performance
(as David says below).  Running XIP from a RAM-based filesystem,
as originally proposed, might solve the performance problem,
but it wouldn't improve space utilization, and since you have to
populate the RAM from somewhere on startup anyway, I don't think
it will improve your bootup time.  It _might_ improve per-application
startup time, once the system is running, but I think a regular
RAM disk suffices for that.

> You may get a _slightly_ faster startup time after reboot if you use XIP
> from flash, because it doesn't have to be loaded into RAM first. But
> that comes at the cost of making it all a low slower during normal
> operation -- it's a lot slower to fetch icache lines from flash than it
> is from RAM.

FYI - Here are some rough numbers:
Time to run shell script which starts TinyX X server and "xsetroot -solid red",
then shuts down:

First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds
Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds

I think this was on a 133 MHz PPC, but I'm not positive.  In both cases
the filesystem was in flash.  Note that once the application pages are
in RAM in the page cache, the Non-XIP case beats the XIP case (probably
due to the penalty to access flash memory).

So the only performance win is on the first invocation of the application.

I'm just now starting to put together a wiki page of information about
Application XIP, for those interested. (It has nothing now, so don't get
excited.)  But contributions are welcome at:
http://tree.celinuxforum.org/pubwiki/moin.cgi/ApplicationXIP
(Maybe when someone googles this message a year from now, there will be
something interesting there... ;-)

=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: tim.bird@am.sony.com
=============================

Re: What File System supports Application XIP

[Todd Poynor]

Tim Bird wrote:
...
> The patches I've seen require setting the CRAMFS_LINEAR option, to turn on
> linear addressing for cramfs, and CRAMFS_LINEAR_XIP.  The result of these
> is to dispense with compression.

Compression is skipped for the XIP files, which are typically marked via 
the sticky bit.  You'll also need a version of mkcramfs that creates the 
image without compressing those files.

...
> FYI - Here are some rough numbers:
> Time to run shell script which starts TinyX X server and "xsetroot -solid red",
> then shuts down:
> 
> First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds
> Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds
> 
> I think this was on a 133 MHz PPC, but I'm not positive.  In both cases
> the filesystem was in flash.  

It was measured on a 168MHz ARM 925T TI OMAP 1510.

Others' advice that "you probably don't want XIP" is true in most cases. 
  But in producing a battery-operated product with certain requirements 
for performance, power savings (due to reduced RAM requirements), 
startup time (depending on the platform and software stack the 
difference can be significant), etc. XIP is an option chosen by some CE 
designers, who are willing to accept the performance penalty on a 
product that will still run adequately for its intended uses.  It would 
be interesting to see an in-depth analysis of these topics on an actual 
Linux-based product such as a cell phone.  There are, of course, a 
number of ways to address all these issues, not just XIP...

-- 
Todd Poynor
MontaVista Software

Re: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP

[colin]

Hi Paulo,
Cramfs seems to support it now.
At the beginning, I also think it's impossible to support Application XIP in
Cramfs.
http://www.denx.de/twiki/publish/DULG/ConfigureLinuxForXIP.html

Regards,
Colin


----- Original Message ----- 
From: "Paulo Marques" <pmarques@grupopie.com>
To: "colin" <colin@realtek.com.tw>
Cc: <linux-kernel@vger.kernel.org>
Sent: Thursday, September 09, 2004 5:19 PM
Subject: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP


> colin wrote:
> >
> > Hi there,
> > We are developing embedded Linux system. Performance is our
consideration.
> > We hope some applications can run as fast as possible,
> > and are think if they can be put in a filesystem image, which resides in
> > RAM, and run in XIP (eXecute In Place)  manners.
> > I know that Cramfs has supported Application XIP. Is there any other FS
that
> > also supports it? Ramdisk? Ramfs? Romfs?
>
> Obvisously cramfs can not support XIP, because the "in-place" image
> is compressed (unless you have a processor that can execute compressed
> code :)
>
> AFAIK only tmpfs supports XIP because it works on a higher level
> without using block devices underneath. Ramdisks are simply RAM
> block devices that behave like any other block device.
>
> You can have a compressed image in flash (for instance), decompress
> everything into a tmpfs and execute from there.
>
> I'm not sure, however, that this will be such a performance gain.
>
> If you use cramfs (for instance) then the kernel will uncompress
> and run only the pages that are needed, and they will be cached in
> page cache so that they will be available again when needed. This
> way you only waste the RAM you actually need, and can still drop
> old pages if the application needs more RAM.
>
> Just my two cents,
>
> -- 
> Paulo Marques - www.grupopie.com
>
> To err is human, but to really foul things up requires a computer.
> Farmers' Almanac, 1978
>

Re: What File System supports Application XIP

[Arnd Bergmann]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

On Donnerstag, 9. September 2004 10:55, colin wrote:
> I know that Cramfs has supported Application XIP. Is there any other FS that
> also supports it? Ramdisk? Ramfs? Romfs?

On http://linuxvm.org/Patches/, you can find a file system called xip2fs,
that uses an ext2 read-only fs for XIP. The code there works only if the
backing memory is a zSeries DCSS memory segment, but it should be fairly
easy to port to some other low-level memory provider.

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]