[RFC/PATCH 0/5] add execute in place support

[RFC/PATCH 0/5] add execute in place support

[Carsten Otte]

Folks,

this is the intro to a small series of patches that introduce execute
in place into the I/O stack. File I/O to memory-backed block
devices is performed directly, bypassing the page cache and io
schedulers. On s390, we use this for block devices based on shared
memory between multiple virtual machines. This is also useful on
embedded systems where the block device is located on a flash chip.
This work is a result of a prior discussion with Andrew Morton
about my first implementation which basically was a filesystem
derived from ext2.

As I'd like to aim for integration into -mm and vanilla later on,
I'd like to encourage everyone to give it a read and provide
feedback. All patches apply against git-head as of today.

cheers,
Carsten

Re: [RFC/PATCH 0/5] add execute in place support

[David Woodhouse]

On Wed, 2005-05-11 at 16:29 +0200, Carsten Otte wrote:
> . This is also useful on embedded systems where the block device is
> located on a flash chip.

On Wed, 2005-05-11 at 17:33 +0200, Carsten Otte wrote:
> Indeed that seems reasonable. There is no exact reason to have
> this built into a kernel on a platform that does not have a bdev
> for this.

The sanest way to use flash chips is not to pretend that they're a block
device at all; rather to use a file system directly on top of them. 

But although you _talk_ about block devices, your code does look like it
should be usable even by flash file systems. I'll try to come up with a
test case using it on flash.

-- 
dwmw2

Re: [RFC/PATCH 0/5] add execute in place support

[Carsten Otte]

David Woodhouse wrote:

>On Wed, 2005-05-11 at 16:29 +0200, Carsten Otte wrote:
>  
>
>>. This is also useful on embedded systems where the block device is
>>located on a flash chip.
>>    
>>
>
>On Wed, 2005-05-11 at 17:33 +0200, Carsten Otte wrote:
>  
>
>>Indeed that seems reasonable. There is no exact reason to have
>>this built into a kernel on a platform that does not have a bdev
>>for this.
>>    
>>
>
>The sanest way to use flash chips is not to pretend that they're a block
>device at all; rather to use a file system directly on top of them. 
>
>But although you _talk_ about block devices, your code does look like it
>should be usable even by flash file systems. I'll try to come up with a
>test case using it on flash.
>
>  
>
Yes and no. For execute in place to work proper, you need an
allignment of data to page boundaries "on disk" (or on flash)
just like you have when mmap()ing to userland.
Before choosing second extended, I also looked at some
flash/rom filesystems. But I was unable to identify one that
alligns the data proper (and does not compress things or
such). The ext family with block size == PAGE_SIZE does
fullfill that requirement once the "block device" starts on page
boundary.
On the other hand I believe that a filesystem specificaly
designed for flash can provide less metadata overhead then
second extended. Would also be interresting in our use-case
on s390.

Re: [RFC/PATCH 0/5] add execute in place support

[Jörn Engel]

On Wed, 11 May 2005 18:35:28 +0200, Carsten Otte wrote:
> >
> Yes and no. For execute in place to work proper, you need an
> allignment of data to page boundaries "on disk" (or on flash)
> just like you have when mmap()ing to userland.
> Before choosing second extended, I also looked at some
> flash/rom filesystems. But I was unable to identify one that
> alligns the data proper (and does not compress things or
> such). The ext family with block size == PAGE_SIZE does
> fullfill that requirement once the "block device" starts on page
> boundary.
> On the other hand I believe that a filesystem specificaly
> designed for flash can provide less metadata overhead then
> second extended. Would also be interresting in our use-case
> on s390.

In principle, both the block device abstraction and the mtd
abstraction fit your bill.  But jffs2 doesn't, so no in-kernel fs
could make use of a xip-aware mtd abstraction.

Patching jffs2 for xip looks like a major effort, at best, and utterly
insane at worst.  I'd prefer not to go down that path.

Jörn

-- 
Optimizations always bust things, because all optimizations are, in
the long haul, a form of cheating, and cheaters eventually get caught.
-- Larry Wall 
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Carsten Otte]

Jörn Engel wrote:

> In principle, both the block device abstraction and the mtd
>
>abstraction fit your bill.  But jffs2 doesn't, so no in-kernel fs
>could make use of a xip-aware mtd abstraction.
>  
>
True. In fact I thought about an mtd device driver for our dcss
segments instead of a block device driver. A filesystem on top
of mtd that implements get_xip_page does the very same job.
But after I decided to build on the ext family, I discarded that
idea because ext does use the block device interface.


-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[David Woodhouse]

On Thu, 2005-05-12 at 10:57 +0200, Jörn Engel wrote:
> In principle, both the block device abstraction and the mtd
> abstraction fit your bill.  But jffs2 doesn't, so no in-kernel fs
> could make use of a xip-aware mtd abstraction.
> 
> Patching jffs2 for xip looks like a major effort, at best, and utterly
> insane at worst.  I'd prefer not to go down that path.

You and me both. The time has definitely come to recognise that JFFS2
needs replacing ;)

-- 
dwmw2

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Coywolf Qi Hunt]

On 5/12/05, David Woodhouse <dwmw2@infradead.org> wrote:
> On Thu, 2005-05-12 at 10:57 +0200, Jörn Engel wrote:
> > In principle, both the block device abstraction and the mtd
> > abstraction fit your bill.  But jffs2 doesn't, so no in-kernel fs
> > could make use of a xip-aware mtd abstraction.
> >
> > Patching jffs2 for xip looks like a major effort, at best, and utterly
> > insane at worst.  I'd prefer not to go down that path.
> 
> You and me both. The time has definitely come to recognise that JFFS2
> needs replacing ;)

I'd say yaffs seems to be a good one. 

-- 
Coywolf Qi Hunt
http://ahbl.org/~coywolf/
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Coywolf Qi Hunt]

On 5/11/05, Carsten Otte <cotte@de.ibm.com> wrote:
> Folks,
> 
> this is the intro to a small series of patches that introduce execute
> in place into the I/O stack. File I/O to memory-backed block
> devices is performed directly, bypassing the page cache and io
> schedulers. On s390, we use this for block devices based on shared
> memory between multiple virtual machines. This is also useful on
> embedded systems where the block device is located on a flash chip.
> This work is a result of a prior discussion with Andrew Morton
> about my first implementation which basically was a filesystem
> derived from ext2.
> 
> As I'd like to aim for integration into -mm and vanilla later on,
> I'd like to encourage everyone to give it a read and provide
> feedback. All patches apply against git-head as of today.

I feel the name "execute in place" misleading. This is not the real
XIP, IMHO. Invent another term or be tolerant?
-- 
Coywolf Qi Hunt
http://ahbl.org/~coywolf/

Re: [RFC/PATCH 0/5] add execute in place support

[Jörn Engel]

On Wed, 1 June 2005 17:33:29 +0800, Coywolf Qi Hunt wrote:
> On 5/12/05, David Woodhouse <dwmw2@infradead.org> wrote:
> > On Thu, 2005-05-12 at 10:57 +0200, Jörn Engel wrote:
> > > In principle, both the block device abstraction and the mtd
> > > abstraction fit your bill.  But jffs2 doesn't, so no in-kernel fs
> > > could make use of a xip-aware mtd abstraction.
> > >
> > > Patching jffs2 for xip looks like a major effort, at best, and utterly
> > > insane at worst.  I'd prefer not to go down that path.
> > 
> > You and me both. The time has definitely come to recognise that JFFS2
> > needs replacing ;)
> 
> I'd say yaffs seems to be a good one. 

Not if you care about XIP.  Yaffs won't run on NOR flashes and XIP
won't work on NAND.

Jörn

-- 
When you close your hand, you own nothing. When you open it up, you
own the whole world.
-- Li Mu Bai in Tiger & Dragon
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Arnd Bergmann]

On Middeweken 01 Juni 2005 12:12, Coywolf Qi Hunt wrote:
> > 
> > As I'd like to aim for integration into -mm and vanilla later on,
> > I'd like to encourage everyone to give it a read and provide
> > feedback. All patches apply against git-head as of today.
> 
> I feel the name "execute in place" misleading. This is not the real
> XIP, IMHO. Invent another term or be tolerant?

If this is not real execute in place, then what is? The term seems to
describe the patch rather well and we don't have this ability for Linux
applications anywhere else, at least not in official kernels.

	Arnd <><

Re: [RFC/PATCH 0/5] add execute in place support

[Jörn Engel]

On Wed, 1 June 2005 14:06:03 +0200, Arnd Bergmann wrote:
> On Middeweken 01 Juni 2005 12:12, Coywolf Qi Hunt wrote:
> > > 
> > > As I'd like to aim for integration into -mm and vanilla later on,
> > > I'd like to encourage everyone to give it a read and provide
> > > feedback. All patches apply against git-head as of today.
> > 
> > I feel the name "execute in place" misleading. This is not the real
> > XIP, IMHO. Invent another term or be tolerant?
> 
> If this is not real execute in place, then what is? The term seems to
> describe the patch rather well and we don't have this ability for Linux
> applications anywhere else, at least not in official kernels.

We can execute the kernel in place, thanks to Nicolas Pitre.  For
userspace I have seen to preparation patches, but nothing serious.

Jörn

-- 
I've never met a human being who would want to read 17,000 pages of
documentation, and if there was, I'd kill him to get him out of the
gene pool.
-- Joseph Costello
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Carsten Otte]

Jörn Engel wrote:

>On Wed, 1 June 2005 14:06:03 +0200, Arnd Bergmann wrote:
>  
>
>>On Middeweken 01 Juni 2005 12:12, Coywolf Qi Hunt wrote:
>>    
>>
>>>>As I'd like to aim for integration into -mm and vanilla later on,
>>>>I'd like to encourage everyone to give it a read and provide
>>>>feedback. All patches apply against git-head as of today.
>>>>        
>>>>
>>>I feel the name "execute in place" misleading. This is not the real
>>>XIP, IMHO. Invent another term or be tolerant?
>>>      
>>>
>>If this is not real execute in place, then what is? The term seems to
>>describe the patch rather well and we don't have this ability for Linux
>>applications anywhere else, at least not in official kernels.
>>    
>>
>
>We can execute the kernel in place, thanks to Nicolas Pitre.  For
>userspace I have seen to preparation patches, but nothing serious.
>  
>
Given that execute in place describes the ability to run code at
the very same location where it is permanently stored, and that
this is exactly what the patch enables, I think the wording is
correct.


-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH 0/5] add execute in place support

[Jörn Engel]

On Wed, 1 June 2005 16:03:45 +0200, Carsten Otte wrote:
> Jörn Engel wrote:
> 
> >We can execute the kernel in place, thanks to Nicolas Pitre.  For
> >userspace I have seen to preparation patches, but nothing serious.
  *)
> >
> Given that execute in place describes the ability to run code at
> the very same location where it is permanently stored, and that
> this is exactly what the patch enables, I think the wording is
> correct.

*) except for Carsten's patches.  Sorry about the confusion.

Jörn

-- 
This above all: to thine own self be true.
-- Shakespeare
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html