Unable to mount compressed root f/s because init_mtd() does not r egister block device!

Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[Vipin Malik]

In 2.2.18 patched with the 2.2.18 patch, I'm trying to boot
a kernel from raw (CFI) flash and decompress an ext2 root
f/s image (from /dev/mtdblock0) into ramdisk and mount it as root.

Everything goes fine, (kernel boots etc.) but when the load ramdisk
code runs, it fails with a -ENODEV when it tries to open
the root device (to load the compressed root file system, present
on /dev/mtdblock0- right after the kernel).

It fails because, even though init_mtd() has been already
called before the rd_load() call in init/main.c, init_mtd()
does NOT call init_mtdblock() which actually registers the
mtdblock device.
This is done much later when the file system is actually installed.

Why? Any particular reason or no one tried this before.

Thanks

Vipin


To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[Vipin Malik]

Bacause, the ram disk driver needs a registered block device to
decompress from. This code already exists in the linux kernel (see
drivers/block/rd.c)
(to uncompress a compressed root file system image). I don't want to
write it again.

Also, as far as reading goes, there is no real difference between the
mtd0 device and the block device (i.e. reading from
mtd0 does not get you anything extra).

Vipin



Amit D Chaudhary wrote:

> Hi,
>
> A question here, why not uncompress from the raw mtd device(that is
> /dev/mtd0, /dev/mtd1, etc) instead?
>
> Thanks
> Amit
>
> Vipin Malik wrote:
>
> > In 2.2.18 patched with the 2.2.18 patch, I'm trying to boot
> > a kernel from raw (CFI) flash and decompress an ext2 root
> > f/s image (from /dev/mtdblock0) into ramdisk and mount it as root.
> >



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[David Woodhouse]

vmalik@danielind.com said:
>  Also, as far as reading goes, there is no real difference between the
> mtd0 device and the block device (i.e. reading from mtd0 does not get
> you anything extra).

Except that the block device is quite a lot of unnecessary overhead (and a 
kernel thread, etc.). Reading directly from the MTD device rather than 
through either the blockdevice or chardevice would be nicer. 

Actually, for 2.4 I'd like to be able to extract a tarball from an MTD 
device directly into ramfs rather than ramdisk. CONFIG_BLK_DEV=n


--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does n ot r egister block device!

[David Woodhouse]

On Thu, 22 Feb 2001, Vipin Malik wrote:

> Success!! Well sort of ;)
> 
> After a terrible hack of explicitly calling rd_load() again at the end
> of init_mtdblock(), forces the ramdisk stuff to look for the
> compressed root file system again. This time it finds it (now that
> mtdblock has registered the block device) enabling the root file
> system to be loaded and be mounted!

Heh. Well done.

> This proves for sure that that is the only issue left to boot
> compressed root file systems off raw flash. Once we figure out how to
> force init_mtdblock() to be linked in before the ram disk stuff, we
> would be set.

Cool. You'll have to hurry, though - We'll have JFFS2 (with compression) 
ready to roll out quite soon :)

-- 
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[Vipin Malik]

David Woodhouse wrote:

> On Thu, 22 Feb 2001, Vipin Malik wrote:
>
> > Success!! Well sort of ;)
> >
> > After a terrible hack of explicitly calling rd_load() again at the end
> > of init_mtdblock(), forces the ramdisk stuff to look for the
> > compressed root file system again. This time it finds it (now that
> > mtdblock has registered the block device) enabling the root file
> > system to be loaded and be mounted!
>
> Heh. Well done.

Thanks.

>
>
> > This proves for sure that that is the only issue left to boot
> > compressed root file systems off raw flash. Once we figure out how to
> > force init_mtdblock() to be linked in before the ram disk stuff, we
> > would be set.
>
> Cool. You'll have to hurry, though -

Are you saying that you'll not fix this "bug" for me ;)
Would saying please help :)

> We'll have JFFS2 (with compression)
>ready to roll out quite soon :)

Well, there still would be reasons to decompress a root file system into
ramdisk and run from there, namely:

1. Processor cache: Most likely you would have turned off the processor
cache on the flash window, to enable writing.
A cached (or possibly even uncached) DRAM would be faster than
FLASH (specially if one saved some money and
bought those 120ns ones :) Additionally many high end embedded processors
have (or will have in the future), write merging and
read ahead that you want turned off again so as not to screw up write and
erase cycles.

2. Reliability: A wayward program is less likely to completely screw up the
system. A reboot would rebuild the root file system
to a pristine state.

3. Compressibility: I would hazard a guess that the compressibility
obtained with compressing an entire file system image would
be better than that obtained with just compressing individual inode data.

4. For faster access times, you want the flash connected across the entire
bus data width. Say using x8 chips with 64KB sectors,
and 2 sectors min erase area, you are looking at 512KBytes of non usable
flash space. This is less of an issue with compressed fs!
One can connect just a x8 chip and have the performance of running out of
dram and the small overhead of 2*64=128KBytes erase
buffer available.

5. Compressed root file systems will cook and do your laundry for you, will
jffs2 do that?

:)

Vipin

>
>
> --
> dwmw2



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[David Woodhouse]

vmalik@danielind.com said:
>  Are you saying that you'll not fix this "bug" for me ;) Would saying
> please help :) 

Heh. Of course I'll help. Not sure how best to go about it, though. Init 
order dependencies are ugly. rd_load() should probably be one of the last 
things in the boot sequence, rather than an initcall. Can you add it to 
init/main.c just before mount_root()?



> > We'll have JFFS2 (with compression) ready to roll out quite soon :)

> Well, there still would be reasons to decompress a root file system
> into ramdisk and run from there, namely: 

Definitely. Just pushing my new baby, that's all :)


>  3. Compressibility: I would hazard a guess that the compressibility
> obtained with compressing an entire file system image would be better
> than that obtained with just compressing individual inode data. 

Yep. Actually we only compress a single page at a time 
in jffs2, not even the whole inode.


--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[Vipin Malik]

David Woodhouse wrote:

> vmalik@danielind.com said:
> >  Are you saying that you'll not fix this "bug" for me ;) Would saying
> > please help :)
>
> Heh. Of course I'll help. Not sure how best to go about it, though. Init
> order dependencies are ugly. rd_load() should probably be one of the last
> things in the boot sequence, rather than an initcall. Can you add it to
> init/main.c just before mount_root()?

Ok, I did it in my kernel and that worked too, but that's not the permanent
solution is it?


>
>
> > > We'll have JFFS2 (with compression) ready to roll out quite soon :)
>
> > Well, there still would be reasons to decompress a root file system
> > into ramdisk and run from there, namely:
>
> Definitely. Just pushing my new baby, that's all :)
>
> >  3. Compressibility: I would hazard a guess that the compressibility
> > obtained with compressing an entire file system image would be better
> > than that obtained with just compressing individual inode data.
>
> Yep. Actually we only compress a single page at a time
> in jffs2, not even the whole inode.

Do you have any figures as to how well it compresses vs say gzip -9?

Do you know why nobody added bzip2 support for compressed root file systems?
bzip2
does a better job at compression. Size of the code?

Vipin

>
>
> --
> dwmw2
>
> To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Unable to mount compressed root f/s because init_mtd() does not r egister block device!

[David Woodhouse]

vmalik@danielind.com said:
>  Do you have any figures as to how well it compresses vs say gzip -9?

Not wonderfully, especially if you're compressing something other than 
MIPS32 executable code. But it's easy enough to plug in new compression 
types. gzip would be welcome.

> Do you know why nobody added bzip2 support for compressed root file
> systems? bzip2 does a better job at compression. Size of the code? 

scratch space required to decompress

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org