logfs: max 4K writepage size

logfs: max 4K writepage size

[srimugunthan dhandapani]

Hi,

>
>> While i still have your attention, i would like to point out at max
>> writepage size restriction in logfs. Currently logfs has a max
>> writepage size as 4K. Currently, large page nand flashes and MLC nand
>> come in  8K page sizes. As the usecase for logfs is large nand
>> flashes(mostly with parallel and DMA write capabilities),  it may be
>> essential to remove the 4K write page size restriction.
>> I was only able to change the logfs-tools for >4K writepage size. If
>> you send a patch, that makes logfs useable for writepagesize >4K, i
>> can try on real hardware, instead of nandsim and give you the results
>> :-)
>
> That sure sounds useful.  I'll have a look...
>
> Jörn

If you plan to do this, when can we expect your patches in the kernel?
Can you suggest what changes have to be done to have >4K writepage size.
>From what i looked, it doesnt seem straight forward. I think the 4K
writepage size restriction is because
the flash device is memory mapped for caching purposes.
Initially, I wouldnt want to have the caching feature. Is there some
code in the ancient logfs tree
(http://git.kernel.org/?p=linux/kernel/git/joern/ancient_logfs.git;a=summary)
without the device mapping, that i can make use of?

Thanks,
mugunthan

Re: logfs: max 4K writepage size

[Jörn Engel]

On Sun, 4 September 2011 15:50:03 +0530, srimugunthan dhandapani wrote:
> 
> If you plan to do this, when can we expect your patches in the kernel?

Don't expect me to predict the future - my past predictions have not
proven to be very reliable.

> Can you suggest what changes have to be done to have >4K writepage size.

I think the only change strictly necessary is the patch below,
removing an assertion.  Plus the second patch below for mklogfs.

> From what i looked, it doesnt seem straight forward. I think the 4K
> writepage size restriction is because
> the flash device is memory mapped for caching purposes.
> Initially, I wouldnt want to have the caching feature.

Careful.  I know that for some devices the caching makes a performance
difference somewhere between 100x and 1000x.  Pretty much whenever you
encounter a crap FTL on your random USB stick, SDcard, etc. that is
the case.  So if you want to avoid caching for your purposes, you'd
have to do it in a way that doesn't cause a huge performance
regression to these devices.  In other words, caching needs to stay in
the code, but be made contingent on some condition that I couldn't
specify in half an hour.

As the result - having both caching and non-caching code, plus some
decision heuristic - will be a non-trivial maintenance burden, there
should also be a non-trivial performance benefit attached.  But then
again, I suppose the two patches below mean you won't even attempt
going non-caching anyway. :)

Jörn

-- 
Unless something dramatically changes, by 2015 we'll be largely
wondering what all the fuss surrounding Linux was really about.
-- Rob Enderle

[PATCH] logfs: remove useless BUG_ON

It prevents write sizes >4k.

Signed-off-by: Joern Engel <joern@logfs.org>
---
 fs/logfs/journal.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/fs/logfs/journal.c b/fs/logfs/journal.c
index 9da2970..1e1c369 100644
--- a/fs/logfs/journal.c
+++ b/fs/logfs/journal.c
@@ -612,7 +612,6 @@ static size_t __logfs_write_je(struct super_block *sb, void *buf, u16 type,
 	if (len == 0)
 		return logfs_write_header(super, header, 0, type);
 
-	BUG_ON(len > sb->s_blocksize);
 	compr_len = logfs_compress(buf, data, len, sb->s_blocksize);
 	if (compr_len < 0 || type == JE_ANCHOR) {
 		memcpy(data, buf, len);
-- 
1.7.2.3


[PATCH] Allow larger write shift

Current flashes with 8k write size already exist.  Why pick 16?  No
good reason, it's a bit bigger and will do for a while.  Maybe 32 or
64 would be sane choices - beyond 64 is definitely insane - but until
someone can properly argue where exactly the boundary should be, this
is good enough for a while.

Signed-off-by: Joern Engel <joern@logfs.org>
---
 mkfs.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/mkfs.c b/mkfs.c
index fd54b75..138067a 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -514,8 +514,8 @@ static void mkfs(struct super_block *sb)
 		fail("segment shift must be larger than block shift");
 	if (blockshift != 12)
 		fail("blockshift must be 12");
-	if (writeshift > 12)
-		fail("writeshift too large (max 12)");
+	if (writeshift > 16)
+		fail("writeshift too large (max 16)");
 	sb->segsize = 1 << segshift;
 	sb->blocksize = 1 << blockshift;
 	sb->blocksize_bits = blockshift;
-- 
1.7.2.3

Re: logfs: max 4K writepage size

[srimugunthan dhandapani]

On Wed, Sep 7, 2011 at 12:26 PM, Jörn Engel <joern@logfs.org> wrote:
>
>> Can you suggest what changes have to be done to have >4K writepage size.
>
> I think the only change strictly necessary is the patch below,
> removing an assertion.  Plus the second patch below for mklogfs.
>

Thanks for the patches.
I was able to do  basic mount-copy-unmount once.
 Let me test more with multiple mount unmount cycles and see if there
are problems.
I think we have to give mklogfs /dev/mtdblock0 instead of /dev/mtd0
I updated wikipedia with steps to use logfs
http://en.wikipedia.org/wiki/LogFS
thanks,
mugunthan

Re: logfs: max 4K writepage size

[Jörn Engel]

On Wed, 7 September 2011 23:00:30 +0530, srimugunthan dhandapani wrote:
> 
> I was able to do  basic mount-copy-unmount once.
>  Let me test more with multiple mount unmount cycles and see if there
> are problems.

I'm always happy when I can fix bugs.  Though not always happy to
discover that the d*ckhead who created them was me.

> I think we have to give mklogfs /dev/mtdblock0 instead of /dev/mtd0

You can use mtd0 instead of /dev/mtd0, similar to jffs2.  Any /dev/foo
gets interpreted by some common code in the kernel.  As it's not a
block device, mount fails.  mtd0 doesn't start with /dev/, so it must
be special somehow and gets passed to the filesystem.  Jffs2 and logfs
are can then do something with this string.

And yes, this sounds really stupid.  Having /dev/mtd0 do the right
thing would certainly be less confusing than the current state.

> I updated wikipedia with steps to use logfs
> http://en.wikipedia.org/wiki/LogFS

Cool!  Thanks!

Jörn

-- 
Schrödinger's cat is <BLINK>not</BLINK> dead.
-- Illiad

Re: logfs: max 4K writepage size

[srimugunthan dhandapani]

On Thu, Sep 8, 2011 at 12:47 AM, Jörn Engel <joern@logfs.org> wrote:

> You can use mtd0 instead of /dev/mtd0, similar to jffs2.

With logfs, I am a bit confused about when ultimately the data is
written to the flash and in what units it is written as.
The mtd_write() function in dev_mtd.c does writes in units of 4096 bytes.
With Nand flash writepagesize >4K, using "mklogfs mtd0 & mount mtd0" ,
won't logfs  pass non-flashpage aligned writes to the nand flash
driver?.

thanks,
mugunthan