MTD, generic physmap memory, MTD_BLOCK

MTD, generic physmap memory, MTD_BLOCK

[Michael Stumpf]

I'm working in embedded LinuxPPC on a system with low RAM and ROM -- 6 meg RAM,
8 meg flash ROM.  It seems like that MTD may provide (or almost so) a method
for mapping memory space in as a block device.  It already appears to sort-of
do that, but I was wondering if full support for booting off of non-CFI
compliant, but transparently mapped in, memories is either in place now
(doesn't seem to be there) or is planned for the future.

It's a trivial driver, and I'll write it / modify existing MTD code if
desired/needed, but it really seems important to have so that I can
execute-in-place from a cramfs partition in ROM (and also have this as my root
partition).

Thoughts?  Am I way off the mark here?

Regards-
Michael Stumpf



=====
=====================================================================
Michael J. Stumpf                              
LinuxPPC enablement
IBM-Austin, TX     512.838.1524 [lab]  512.838.5335 [work]   
http://www.pobox.com/~mstumpf

__________________________________________________
Do You Yahoo!?
Yahoo! Shopping - Thousands of Stores. Millions of Products.
http://shopping.yahoo.com/


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Nicolas Pitre]

On Wed, 13 Dec 2000, Michael Stumpf wrote:

>
> I'm working in embedded LinuxPPC on a system with low RAM and ROM -- 6 meg RAM,
> 8 meg flash ROM.  It seems like that MTD may provide (or almost so) a method
> for mapping memory space in as a block device.  It already appears to sort-of
> do that, but I was wondering if full support for booting off of non-CFI
> compliant, but transparently mapped in, memories is either in place now
> (doesn't seem to be there) or is planned for the future.

Look at mtdram.c.  It might require verylittle modifications to do just what
you want.

> It's a trivial driver, and I'll write it / modify existing MTD code if
> desired/needed, but it really seems important to have so that I can
> execute-in-place from a cramfs partition in ROM (and also have this as my root
> partition).

It's impossible to execute_in_place from a cramfs partition since the
in-place code is compressed.


Nicolas



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Shane Nay]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]


> > It's a trivial driver, and I'll write it / modify existing MTD code if
> > desired/needed, but it really seems important to have so that I can
> > execute-in-place from a cramfs partition in ROM (and also have this as my
> > root partition).
>
> It's impossible to execute_in_place from a cramfs partition since the
> in-place code is compressed.

Unless you expand the inode and mark things as uncompressed and aligned and 
others as compressed.  Attached is a patch to cramfs that allows it to access 
the memory locations that are mapped in directly rather than buffering 
compressed data..., which is a big fat waste and is a "first step" towards 
XIPable cramfs.  We're working on a cramfs that is XIPable, but only for 
MIPs.  The increased performance of a linearly addressed cramfs partition is 
pretty decent..., although I haven't figured out a smart way to bootstrap 
right into the cramfs stuff...  (I'm thinking init.c or around there is where 
I should be looking)

Thanks,
Shane.

Oh, BTW, I finished "a version" of mtd for XIP kernel writable for 
cfi_cmdset_0001.c, and cfi_probe.  I haven't figured out a smart way to deal 
with having two versions of stuff yet..., because most of the code is the 
same.  You can find it on 
ftp.agendacomputing.com/pub/kernel/*someversion*/patches .  Once I figure out 
a smart way to do this, and forward port to CVS I'll put it in the mainline 
MTD code unless there are any nay-sayers.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: cramfs-linear-addressing.patch --]
[-- Type: text/english; name="cramfs-linear-addressing.patch", Size: 2826 bytes --]

diff -urN linux.orig/fs/Config.in linux/fs/Config.in
--- linux.orig/fs/Config.in	Fri Oct 27 04:23:18 2000
+++ linux/fs/Config.in	Fri Oct 27 03:57:52 2000
@@ -29,6 +29,10 @@
 	int 'JFFS debugging verbosity (0 = quiet, 3 = noisy)' CONFIG_JFFS_FS_VERBOSE 0
 fi
 tristate 'Compressed ROM file system support' CONFIG_CRAMFS
+dep_mbool 'Linear addressing for CRAMFS' CONFIG_CRAMFS_LINEAR_ADDRESSING $CONFIG_CRAMFS
+if [ "$CONFIG_CRAMFS_LINEAR_ADDRESSING" != "n" ] ; then
+	hex 'Starting address for CRAMFS filesystem' CONFIG_CRAMFS_LA_ADDRESS bf200008
+fi
 tristate 'Simple RAM-based file system support' CONFIG_RAMFS
 
 tristate 'ISO 9660 CDROM file system support' CONFIG_ISO9660_FS
diff -urN linux.orig/fs/cramfs/inode.c linux/fs/cramfs/inode.c
--- linux.orig/fs/cramfs/inode.c	Fri Oct 27 04:22:36 2000
+++ linux/fs/cramfs/inode.c	Fri Oct 27 04:30:18 2000
@@ -11,6 +11,20 @@
  * The actual compression is based on zlib, see the other files.
  */
 
+/* Linear Addressing code
+ *
+ * Copyright (C) 2000 Shane Nay.
+ *
+ * Allows you to have a linearly addressed cramfs filesystem.
+ * Saves the need for buffer, and the munging of the buffer.
+ * Savings a bit over 32k with default PAGE_SIZE, BUFFER_SIZE
+ * etc.  Usefull on embedded platform with ROM :-).
+ *
+ * Downsides- Currently linear addressed cramfs partitions
+ * don't co-exist with block cramfs partitions.
+ *
+ */
+
 #include <linux/module.h>
 #include <linux/fs.h>
 #include <linux/pagemap.h>
@@ -68,6 +82,23 @@
 	return inode;
 }
 
+#if defined(CONFIG_CRAMFS_CRAMFS_LINEAR_ADDRESSING) && defined (CONFIG_CRAMFS_LA_ADDRESS)
+
+/*
+ * Returns a pointer to the linearly addressed filesystem.
+ * Simple byte size pointer addition.
+ */
+static unsigned char* romdisk_top=(unsigned char*) CONFIG_CRAMFS_LA_ADDRESS;
+
+static void *cramfs_read(struct super_block *sb, unsigned int offset, unsigned int len)
+{
+	if (!len)
+		return NULL;
+	return romdisk_top + offset;
+}
+
+#else /* !CONFIG_CRAMFS_LINEAR_ADDRESSING aka Regular block mode */
+
 /*
  * We have our own block cache: don't fill up the buffer cache
  * with the rom-image, because the way the filesystem is set
@@ -149,6 +180,8 @@
 	}
 	return read_buffers[buffer] + offset;
 }
+
+#endif
 			
 
 static struct super_block * cramfs_read_super(struct super_block *sb, void *data, int silent)
@@ -161,10 +194,11 @@
 	set_blocksize(sb->s_dev, PAGE_CACHE_SIZE);
 	sb->s_blocksize = PAGE_CACHE_SIZE;
 	sb->s_blocksize_bits = PAGE_CACHE_SHIFT;

+#if !( defined(CONFIG_CRAMFS_CRAMFS_LINEAR_ADDRESSING) && defined (CONFIG_CRAMFS_LA_ADDRESS) )
 	/* Invalidate the read buffers on mount: think disk change.. */
 	for (i = 0; i < READ_BUFFERS; i++)
 		buffer_blocknr[i] = -1;
+#endif
 
 	/* Read the first block and get the superblock from it */
 	memcpy(&super, cramfs_read(sb, 0, sizeof(super)), sizeof(super));

Re: MTD, generic physmap memory, MTD_BLOCK

[David Woodhouse]

Unless there's something missing (or I'm being dim) that code doesn't do
XIP. It still copies from the cramfs into the page provided to
cramfs_readpage.

What you want to do is make up some struct pages to represent the pages
of the ROM chip and pre-populate the page cache with those pages directly
- marked readonly, of course.

-- 
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Shane Nay]

On Sunday 17 December 2000 22:24, David Woodhouse wrote:
> Unless there's something missing (or I'm being dim) that code doesn't do
> XIP. It still copies from the cramfs into the page provided to
> cramfs_readpage.
>
> What you want to do is make up some struct pages to represent the pages
> of the ROM chip and pre-populate the page cache with those pages directly
> - marked readonly, of course.


Hehe, no, it's not an XIP implementation.  I think I mentioned in my message, 
even if I didn't make it clear (probably not) that this only made cramfs read 
directly off of the flash, taking one layer of abstraction out.  It kills the 
buffer, and the copy to buffer that would usually be associated with a dev 
device, so it speeds up performance of cramfs on a flash device.  It's _only_ 
a performance hack at this point.

But we are working on XIP for cramfs.  But this requires either expanding the 
inode, or stealing a bit or two away from GIDS, or UIDs.  This is only a 
"linear addressing" patch.

Thanks,
Shane.


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[David Woodhouse]

shane@agendacomputing.com said:
>  But we are working on XIP for cramfs.  But this requires either
> expanding the  inode, or stealing a bit or two away from GIDS, or
> UIDs.  This is only a  "linear addressing" patch. 

I was looking at doing it in romfs. You can't have compression anyway, for 
obvious reasons, and it looked a little easier. But I suppose that wouldn't 
allow you to mix XIP-able and compressed files in the same fs.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Shane Nay]

> I was looking at doing it in romfs. You can't have compression anyway, for
> obvious reasons, and it looked a little easier. But I suppose that wouldn't
> allow you to mix XIP-able and compressed files in the same fs.

We *should* have it done this week in romfs.  That's our testbed for the 
first implementation.  I'm going on vacation on Wednesday, but if it's done 
before that I'll point folks to patches on this list.  If not I should have 
them for the next Thursday..., bug me if I don't send it out by then.

Thanks,
Shane.


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Michael Stumpf]

> >  But we are working on XIP for cramfs.  But this requires either
> > expanding the  inode, or stealing a bit or two away from GIDS, or
> > UIDs.  This is only a  "linear addressing" patch. 
> 
> I was looking at doing it in romfs. You can't have compression anyway, for 
> obvious reasons, and it looked a little easier. But I suppose that wouldn't 
> allow you to mix XIP-able and compressed files in the same fs.

I did some work in the direction suggested by (thanks!) Nicolas--modified
mtdram.c to act as a conduit device for plain memory.  I did get a successful
boot from a cramfs rootdisk.  

When I said XIP, I didn't realize that this crowd had a very "serious"
definition of it.  I simply meant I was trying to get around the initrd
buffering issue:  As far as I can tell, up until now the only mechanism that a
CPU (with only RAM and ROM visibility) has at its disposal to boot from is
initrd, that would copy all the ROM image into RAM buffer cache.. and only
understand three file systems (cramfs is not one of them).  I don't mind cramfs
using a buffer page (4k, LinuxPPC) or two to do its work--I was trying to slay
the complete-image-copy daemon.  (My embedded system is 6 meg RAM, 8 meg ROM..
and I want shared libraries, etc)

It seems to me that for the time being and the immediate future, XIP as we're
talking about it would be overkill.  The streamlining and elegance gained in
the operation of XIP is probably offset by the ugliness of implementation and
the potential loss of compression that seems so crucial in embedded devices,
the primary target (right?) of MTD.  I like XIP but I don't think I'd ever use
it.

One very relevant topic that I'd like to discuss is where a generic memory
conduit device belongs.  I would suspect it belongs as a fallback default if
CFI devices can't be probed (I forget the file), but it also might make sense
to give it a separate file for "neatness" sake.  

Cheers,
Michael




=====
=====================================================================
Michael J. Stumpf                              
LinuxPPC enablement
IBM-Austin, TX     512.838.1524 [lab]  512.838.5335 [work]   
http://www.pobox.com/~mstumpf

__________________________________________________
Do You Yahoo!?
Yahoo! Shopping - Thousands of Stores. Millions of Products.
http://shopping.yahoo.com/


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: MTD, generic physmap memory, MTD_BLOCK

[Shane Nay]

> I did some work in the direction suggested by (thanks!) Nicolas--modified
> mtdram.c to act as a conduit device for plain memory.  I did get a
> successful boot from a cramfs rootdisk.
>
> When I said XIP, I didn't realize that this crowd had a very "serious"
> definition of it.  I simply meant I was trying to get around the initrd
> buffering issue:  As far as I can tell, up until now the only mechanism
> that a CPU (with only RAM and ROM visibility) has at its disposal to boot
> from is initrd, that would copy all the ROM image into RAM buffer cache..
> and only understand three file systems (cramfs is not one of them).  I
> don't mind cramfs using a buffer page (4k, LinuxPPC) or two to do its
> work--I was trying to slay the complete-image-copy daemon.  (My embedded
> system is 6 meg RAM, 8 meg ROM.. and I want shared libraries, etc)

It's 32k BTW.  Boot from initrd?..., initial romdisk?..., nope.  Try this 
instead..., /dev/rom.  Patches are on ftp.agendacomputing.com site.  Make a 
cramfs partition with the headers necessary to tag it as a cramfs partition 
(in the scripts/ directory on linux-vr.org ..., mkromfs I think applied to a 
cramfs filesystem)  Then enable /dev/rom, and set it's address at the start 
of the rom.  Then root=/dev/rom as the boot options.

> It seems to me that for the time being and the immediate future, XIP as
> we're talking about it would be overkill.  The streamlining and elegance
> gained in the operation of XIP is probably offset by the ugliness of
> implementation and the potential loss of compression that seems so crucial
> in embedded devices, the primary target (right?) of MTD.  I like XIP but I
> don't think I'd ever use it.

With only 6MB of ram..., it might be a good thing for your device, but I 
don't know what you're planning on running on it.

> One very relevant topic that I'd like to discuss is where a generic memory
> conduit device belongs.  I would suspect it belongs as a fallback default
> if CFI devices can't be probed (I forget the file), but it also might make
> sense to give it a separate file for "neatness" sake.
Thanks,
Shane.


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org