Perf issue with UBIFS

Perf issue with UBIFS

[Bhavesh Parekh]

Hi all,
I am facing performance issue wil using the UBIFS on our platform.

Kernel version 2.6.36
Nand device info : Micron chip with 2K page size & 128K erase-size
ECC : ECC is done by the NAnd-controller.

We have YAFFS2 working on our platform and recently tried UBIFS on our
platform. YAFF2 Perf is 18MB/s and UBIFS is 11.2 MB/s. Looking at the
driver log we found that the UBIFS access pattern to the device is
4096+48 byte read in the main area, which means that there are lots of
unaligned read. Our platform only support aligned read, so we did an
optimization by simplying storing the last read page in the buffer
hopping that on the next read we will be reading the same page and
gain the perf. Doing this we were able to increase the perf to 14.5
MB/s

Is there any way we can make UBI to have the access in the aligned way
only ? Other wise perf will be lesser than YAFFS2.

Further we twick the nand-driver and implemented the page-cache mode
support in the driver. And change the mtd->writesize to 16K so that
request from the upper layer comes in 16K. It work fine with the
YAFFS2 and we see a perf gain of 3-4MB/s but UBIFS request pattern
remains same 4K +48 byte. Is there anything we can do here so that
UBIFS respect mtd->writesize

-Thanks
Bhavesh

Re: Perf issue with UBIFS

[Artem Bityutskiy]

On Mon, 2011-03-07 at 15:54 +0530, Bhavesh Parekh wrote:
> Hi all,
> I am facing performance issue wil using the UBIFS on our platform.
> 
> Kernel version 2.6.36
> Nand device info : Micron chip with 2K page size & 128K erase-size
> ECC : ECC is done by the NAnd-controller.
> 
> We have YAFFS2 working on our platform and recently tried UBIFS on our
> platform. YAFF2 Perf is 18MB/s and UBIFS is 11.2 MB/s.

Compression was disabled I guess?
Are you talking about read speed, right?

>  Looking at the
> driver log we found that the UBIFS access pattern to the device is
> 4096+48 byte read in the main area, which means that there are lots of
> unaligned read.

Yeah, this is data (uncompressed) + data node header.

>  Our platform only support aligned read, so we did an
> optimization by simplying storing the last read page in the buffer
> hopping that on the next read we will be reading the same page and
> gain the perf. Doing this we were able to increase the perf to 14.5
> MB/s

I think this is standard optimization in nand_base.c.
Did you try to enable bulk read by mounting UBIFS with "bulk_read" mount
options (-o bulk_read) ?

> Is there any way we can make UBI to have the access in the aligned way
> only ?

I do not think so. UBIFS has compression, so data node size varies, so
it fundamentally cannot be block-based. YAFFS2 has not compression, thus
it is organized a bit different and it is more "block size aligned",
because YAFFS2 knows that 4KiB of data always take 4KiB on the flash. In
case of UBIFS 4KiB of data may take only 1.5KiB on the flash.

>  Other wise perf will be lesser than YAFFS2.
> 
> Further we twick the nand-driver and implemented the page-cache mode
> support in the driver.

Yeah, that'd be cool.

>  And change the mtd->writesize to 16K so that
> request from the upper layer comes in 16K. It work fine with the
> YAFFS2 and we see a perf gain of 3-4MB/s but UBIFS request pattern
> remains same 4K +48 byte. Is there anything we can do here so that
> UBIFS respect mtd->writesize

Yeah, you can try to do this.

But please, try UBIFS bulk_read option, does it make any difference for
you?

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Perf issue with UBIFS

[Artem Bityutskiy]

On Tue, 2011-03-08 at 14:31 +0200, Artem Bityutskiy wrote:
> But please, try UBIFS bulk_read option, does it make any difference for
> you?

Ah, and of course, use the 'no_chk_data_crc' option to have more fair
comparison!

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Perf issue with UBIFS

[Artem Bityutskiy]

On Thu, 2011-03-10 at 15:39 +0530, Bhavesh Parekh wrote:
> Basically I was testing with Random file/binary file,here compression was enabled.

But then it is not really fair to compare to YAFFS2 which does not spend
time for decompressing.

> >> request from the upper layer comes in 16K. It work fine with the
> >> YAFFS2 and we see a perf gain of 3-4MB/s but UBIFS request pattern
> >> remains same 4K +48 byte. Is there anything we can do here so that
> >> UBIFS respect mtd->writesize
> >
> > Yeah, you can try to do this.
> I tried to increase the UBI_BLOCK_SIZE to 16K, but later found that it can't be greater than system PAGE_SIZE which means it can't be greater than 4K.

Yes, you cannot make it larger.

> For getting any Perf improvement from the page-cache mode, I needed bulk request from the upper layer(>=16K)

Well, bulk-read is doing this, if the file data is not fragmented. But
on fragmented file-system it is difficult to do bulk-reads. You can read
about UBIFS bulk-read here:

http://www.linux-mtd.infradead.org/doc/ubifs.html#L_readahead

Feel free to improve the doc.

> In case of YAFFS2, if I change mtd->writesize to 16K yaffs2 allocation
> unit changed to 16K from default 2K. When I tried the similar change
> in ubifs it didn't work.

Yes, this is right.

> One approach which I am thinking is to have a pool of buffer of 16K,
> and when-ever there is a read-request come read 16K from the flash
> assuming that next read request will be for next page.
> 
> This approach will work fine initially as write for the first time
> will be contiguous and the request from the UBI will be for
> sub-sequent pages. But after a while there will be fragmentation or
> wear-leveling done by the UBI, so request won't be for the subsequent
> pages.
> 
> Is there a way to force writing to contiguous 16K to the flash so that
> advantage from the page-cache mode can be taken

Yes, you can do this with the very latest UBIFS. We have introduced the
notion of max_write_size, and UBIFS tries to write in max_write_size
chunks, if possible. Make max_write_size to be 16K and UBIFS will use
this more.

However, since LEB starts at offset 2KiB, then UBIFS will write in 16KiB
only starting from offset 16KiB. The patches were sent to the MTD ML in
this thread:

http://lists.infradead.org/pipermail/linux-mtd/2011-March/034277.html

They are also in my ubifs-2.6.git tree - you can pick them from there -
and I'm going to send them upstream soon.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Perf issue with UBIFS

[Artem Bityutskiy]

On Thu, 2011-03-17 at 15:14 +0530, Bhavesh Parekh wrote:
> Thanks Artem.
> Patches you mention helped alot. Now the perf is almost same as that
> of YAFFS2.
> 
> I think there will be changes require in the root-fs creation
> tool(ubinize) as well to support the writebufsize. Correct ?

No, I do not think so.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)