RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> You could use a cramfs root filesystem then mount a small 
> jffs2 partition, 

I sometimes get decompression errors from cramfs with a configuration like that.
My guess is that you really shouldn't have a cramfs-partition on the same flash-
chip as a jffs or jffs2 partition since cramfs doesn't use mtd and does not know
if the chip is busy writing or erasing?

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  I sometimes get decompression errors from cramfs with a configuration
> like that. My guess is that you really shouldn't have a
> cramfs-partition on the same flash- chip as a jffs or jffs2 partition
> since cramfs doesn't use mtd and does not know if the chip is busy
> writing or erasing?

The mtdblock driver ought to deal with it correctly. If it doesn't, it's 
broken.

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> The mtdblock driver ought to deal with it correctly. If it 
> doesn't, it's 
> broken.

I'm not using the mtdblock driver for the cramfs partition.
I guess it's me that's broken...

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  I'm not using the mtdblock driver for the cramfs partition. I guess
> it's me that's broken...

What _are_ you using? If you're going through the MTD device for the flash 
chip, calling its read() method, that still ought to work.

If you're trying to just read the compressed data in the flash chip directly
from the memory location you happen to know it's mapped at, you've just
demonstrated why XIP on a writable chips is such a pain :)

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> What _are_ you using? If you're going through the MTD device 
> for the flash 
> chip, calling its read() method, that still ought to work.
> 
> If you're trying to just read the compressed data in the 
> flash chip directly
> from the memory location you happen to know it's mapped at, 
> you've just
> demonstrated why XIP on a writable chips is such a pain :)

The latter i'm afraid... but I'm working on using jffs2 instead of cramfs.
Would you recommend a read-only (mount -o ro) jffs2 root partition as a replacement for a cramfs root partition?

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  The latter i'm afraid... but I'm working on using jffs2 instead of
> cramfs. Would you recommend a read-only (mount -o ro) jffs2 root
> partition as a replacement for a cramfs root partition?

If the mount time is OK for you, yes. Otherwise you'd want checkpointing 
implemented first. 

Note that I actually made it obey 'readonly' quite recently. Until a month 
or so ago, it was still starting the GC thread when it was supposed to be 
RO.

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> If the mount time is OK for you, yes. Otherwise you'd want 
> checkpointing 
> implemented first. 

My partition is fairly small so the mount time is OK right now, but I will
also be using a larger partition (about 3MB) later. Is anybody working on this
checkpointing thing? Could you explain the concept briefly? 

> 
> Note that I actually made it obey 'readonly' quite recently. 
> Until a month 
> or so ago, it was still starting the GC thread when it was 
> supposed to be 
> RO.

It has worked fine for me this far.

Is it possible to "hide" a bootloader at the beginning of a jffs2 partition
so that jffs2 ignores it (when mounting read-only) and still be able to upgrade
the whole partition (bootloader + jffs2 fs) using the mtd character device?

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  My partition is fairly small so the mount time is OK right now, but I
> will also be using a larger partition (about 3MB) later. Is anybody
> working on this checkpointing thing? Could you explain the concept
> briefly?  

Currently we have to scan the whole thing on startup because we know 
nothing about its state. By occasionally writing a checkpoint - a brain 
dump of the jffs2 code's internal state, we can speed up the mount.

You'd only ever write a checkpoint at the beginning of an erase block. 
The mount code would look at the beginning of each block, and would note 
the most recent checkpoint. It can suck in the state from that checkpoint 
and then go on to read every node in the blocks which were written _after_ 
that checkpoint.


>  Is it possible to "hide" a bootloader at the beginning of a jffs2
> partition so that jffs2 ignores it (when mounting read-only) and still
> be able to upgrade the whole partition (bootloader + jffs2 fs) using
> the mtd character device? 

Not really. Why not just upgrade bootloader and jffs2 fs using two separate 
character devices?

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> Currently we have to scan the whole thing on startup because we know 
> nothing about its state. By occasionally writing a checkpoint 
> - a brain 
> dump of the jffs2 code's internal state, we can speed up the mount.
> 
> You'd only ever write a checkpoint at the beginning of an 
> erase block. 
> The mount code would look at the beginning of each block, and 
> would note 
> the most recent checkpoint. It can suck in the state from 
> that checkpoint 
> and then go on to read every node in the blocks which were 
> written _after_ 
> that checkpoint.

I get it, thanks.

> Not really. Why not just upgrade bootloader and jffs2 fs 
> using two separate 
> character devices?

Two reasons:

1. To be able to write to both devices the partitions must start on eraseblock
block boundary. The bootloader I would use with this configuration is less than
10kB and my eraseblocks are 64kB. I would waste to much space on the loader
partition.

2. I would like to keep my old partitioning so I don't have to change the way
upgrade works. And right now I have a partitiontable, bootloader, compressed
kernel and cramfs-image all on the same mtd partition. And mtd is only used for
upgrading the whole lot on that partition.

I would like to put the contents of the cramfs partition and the kernel in a
jffs2 fs instead and modify the bootloader so that it can find the kernel
in the jffs2 fs. And still have the partitiontable and the bootloader at the
beginning of the partition. I was hoping that jffs2 would treat the partition
table and bootloader as garbage and just ignore it.

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  1. To be able to write to both devices the partitions must start on
> eraseblock block boundary. The bootloader I would use with this
> configuration is less than 10kB and my eraseblocks are 64kB. I would
> waste to much space on the loader partition. 

You really don't want JFFS2 erasing and reprogramming the sector with your 
bootloader in, surely?

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> jonas.holmberg@axis.com said:
> >  1. To be able to write to both devices the partitions must start on
> > eraseblock block boundary. The bootloader I would use with this
> > configuration is less than 10kB and my eraseblocks are 64kB. I would
> > waste to much space on the loader partition. 
> 
> You really don't want JFFS2 erasing and reprogramming the 
> sector with your 
> bootloader in, surely?

Wouldn't mounting the partition read-only prevent that?

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  Wouldn't mounting the partition read-only prevent that?

If you have a recent jffs2, yes. Older ones might do some 
garbage-collection anyway.

And if it's _strictly_ readonly then why not use cramfs on /dev/mtdblock1?

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> And if it's _strictly_ readonly then why not use cramfs on 
> /dev/mtdblock1?

I probably should use cramfs, but I still have to put the bootloader in a
separate partition.

Thanks for your help, just one last straw (-:

Is it possible to make the block device use an offset while the character device
doesn't (using an ioctl, or something, on the character device before mounting
the block device) without modifying mtd? Then I could "hide" the bootloader from
the fs in the same partition and still be able to erase and write a new
bootloader and fs using the character device.

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  I probably should use cramfs, but I still have to put the bootloader
> in a separate partition.

Yeah - it might be worthwhile to rethink the partitioning stuff so that you 
can reprogram the whole device at once, and also reread a partition table 
if you're using one.

> Is it possible to make the block device use an offset while the
> character device doesn't (using an ioctl, or something, on the
> character device before mounting the block device) without modifying
> mtd? Then I could "hide" the bootloader from the fs in the same
> partition and still be able to erase and write a new bootloader and fs
> using the character device.

Thinks... there's actually no reason at all why the devices registered by 
your map driver shouldn't overlap - one for the whole chip for 
reprogramming, and one for the cramfs image.

add_mtd_partitions might not like it atm, but we could invent a new flags 
which tells it to accept the overlap.

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> Thinks... there's actually no reason at all why the devices 
> registered by 
> your map driver shouldn't overlap - one for the whole chip for 
> reprogramming, and one for the cramfs image.

Great idea, thanks a lot!

> 
> add_mtd_partitions might not like it atm, but we could invent 
> a new flags 
> which tells it to accept the overlap.

It doesn't complain, so we don't have to invent anything.

One thing, though... (there's always one thing:)
I get a few printk's from drivers/block/ll_rw_blk.c:

attempt to access beyond end of device
1f:03: rw=0, want=992, limit=984
attempt to access beyond end of device
1f:03: rw=0, want=1000, limit=984
attempt to access beyond end of device
1f:03: rw=0, want=1008, limit=984
attempt to access beyond end of device
1f:03: rw=0, want=992, limit=984
attempt to access beyond end of device
1f:03: rw=0, want=1000, limit=984
attempt to access beyond end of device
1f:03: rw=0, want=1008, limit=984

limit=984 seems OK, the size of the new "overlapping" partition is
0xf6000 == (984 * 1024). But I cannot figure out what cramfs is doing
trying to access above that limit.

A comment in drivers/block/ll_rw_blk.c says:

/* This may well happen - the kernel calls bread()
   without checking the size of the device, e.g.,
   when mounting a device. */

Everything seems to work, but the printk's are annoying. Any idea?

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  Everything seems to work, but the printk's are annoying. Any idea?

Does the size of the output from mkcramfs exceed the size of the partition 
the kernel thinks it's in?

--
dwmw2

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> Does the size of the output from mkcramfs exceed the size of 
> the partition 
> the kernel thinks it's in?

No, mkcramfs says: "Everything: 984 kilobytes"

And the image created is 1007616 bytes (984 * 1024).

/Jonas

RE: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[Jonas Holmberg]

> > Does the size of the output from mkcramfs exceed the size of 
> > the partition 
> > the kernel thinks it's in?
> 
> No, mkcramfs says: "Everything: 984 kilobytes"
> 
> And the image created is 1007616 bytes (984 * 1024).

The problem seems to be cramfs assuming 4kB pages (in one lousy if-statement)
and mine are 8kB...

/Jonas

Re: JFFS2 on 8MB Flash-Chip conneted to MPC850 works extremly slo w:(

[David Woodhouse]

jonas.holmberg@axis.com said:
>  The problem seems to be cramfs assuming 4kB pages (in one lousy
> if-statement) and mine are 8kB... 

Yeah, that would manage to screw it quite effectively. There were patches 
around to make cramfs work on Alpha at one point - I don't remember whether 
they made mkcramfs make 8KiB pages or whether they made the cramfs kernel 
code deal with 4KiB cramfs pages. 

--
dwmw2