DiskOnChip write performance

DiskOnChip write performance

[Holger Friedrich]

Hi all,
I'm using a DiskOnChip 2000 and thanks to this mailing list I'm able to
boot linux off it. 
My application needs to write big files pretty fast and I was expecting
a write speed of 500 kBytes/sec, the M-Systems DOC 2000 spec claims to
achieve this. Unfortunately the fastest speed I could obtain was about
70 kBytes/sec which seems pretty poor to me. When I asked the M-Systems
support the answers were not really helpfull so far. 
Does anybody of you guys know, how to speed it up? I'm using ext2 at
present, will another fs be faster?
I would be even glad if someone told me it can't go faster so I can stop
bothering with DOC and try something better.
Thanks in advance for any helpfull hint
Holger
-- 
=========================================================
Holger Friedrich

MAZeT GmbH                  mailto:friedrich@mazet.de
branch office Jena          Phone: +49 3641 280954
Göschwitzer Straße 32
07745 Jena
Germany
Visit our website: http://www.MAZeT.de
=========================================================

Re: DiskOnChip write performance

[David Woodhouse]

friedrich@MAZeT.de said:
> I'm using a DiskOnChip 2000 and thanks to this mailing list I'm able
> to boot linux off it.  My application needs to write big files pretty
> fast and I was expecting a write speed of 500 kBytes/sec, the
> M-Systems DOC 2000 spec claims to achieve this. Unfortunately the
> fastest speed I could obtain was about 70 kBytes/sec which seems
> pretty poor to me. When I asked the M-Systems support the answers were
> not really helpfull so far. 

Are you using their drivers or the real Linux ones? If you're not using
their drivers, I can understand why they couldn't help you very much.

Our driver is fairly naïve, and may well be wasting a lot of time which 
it shouldn't. I can give you pointers on fixing that, if you're 
interested.

--
dwmw2

Re: DiskOnChip write performance

[Ollie Lho]

David Woodhouse wrote:
> 
> 
> Our driver is fairly naïve, and may well be wasting a lot of time which
> it shouldn't. I can give you pointers on fixing that, if you're
> interested.
> 

David,
	Did you "threaded" NFTL code yet ?? Or we still get locked
when NFTL is syncing/flushing ??

Ollie

Re: DiskOnChip write performance

[kira brown]

On Thu, 6 Sep 2001, Holger Friedrich wrote:

> achieve this. Unfortunately the fastest speed I could obtain was about
> 70 kBytes/sec which seems pretty poor to me. When I asked the M-Systems
> support the answers were not really helpfull so far.

The M-Systems drivers are not the topic of this mailing list.

Neither, for that matter, are the M-systems BIOS extensions, or the
patched LILO they supply.

The topic of this list is the Linux MTD layer.  The Linux MTD layer is
*not* written by M-Systems, it is not supplied by them, and their support
staff do not know anything about it.  THe Linux MTD layer does *not* use
the M-systems BIOS extender, the M-systems binary-only linux kernel
driver, or any M-systems code of any other kind.  The M-Systems firmware
has precisely zero to do with this list, the Linux MTD system, or the
JFFS.

Is that understood?  pradeep, this means *you*.

(Sorry David, I had to say it.)

kira.

Re: DiskOnChip write performance

[Herman Oosthuysen]

Hmm, now that wasn't very helpful was it?

Bear in mind that salesmen like to quote the fastest numbers they can get,
which for this type of system would be writing to a freshly formatted, empty
disk.  This speed would be very different from the sustained rate writing to a
disk that needs some recovery/erase functions to be performed.

The erase time of flash chips vary wildly from chip to chip, even in the same
batch of chips.  Some chips may erase in 3 seconds, others in 15 seconds.   The
write time to the chip will likewise vary, but is many orders of magnitude
faster than the erase cycle, so that is not really a factor.  There is nothing
you can do about it, except to ensure that your DOC is clean before you start
writing a whole lot of data to it.

You may be able to speed the system up a little by optimizing the code, but the
improvement will be insignificant compared to the erase cycle time.

Cheers,

Herman
http://www.AerospaceSoftware.com


On Thu, 06 Sep 2001, kira brown wrote:
> On Thu, 6 Sep 2001, Holger Friedrich wrote:
> 
> > achieve this. Unfortunately the fastest speed I could obtain was about
> > 70 kBytes/sec which seems pretty poor to me. When I asked the M-Systems
> > support the answers were not really helpfull so far.
> 
> The M-Systems drivers are not the topic of this mailing list.
> 
> Neither, for that matter, are the M-systems BIOS extensions, or the
> patched LILO they supply.
> 
> The topic of this list is the Linux MTD layer.  The Linux MTD layer is
> *not* written by M-Systems, it is not supplied by them, and their support
> staff do not know anything about it.  THe Linux MTD layer does *not* use
> the M-systems BIOS extender, the M-systems binary-only linux kernel
> driver, or any M-systems code of any other kind.  The M-Systems firmware
> has precisely zero to do with this list, the Linux MTD system, or the
> JFFS.
> 
> Is that understood?  pradeep, this means *you*.
> 
> (Sorry David, I had to say it.)
> 
> kira.
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
-- 
Herman Oosthuysen
E-mail: Herman@WirelessNetworksInc.com
Phone: 403+569-5687, Fax: 403+235-3965

Re: DiskOnChip write performance

[David Woodhouse]

herman@wirelessnetworksinc.com said:
>  You may be able to speed the system up a little by optimizing the
> code, but the improvement will be insignificant compared to the erase
> cycle time.

Don't count on that if you're using my drivers. There are some fairly 
inefficient algorithms in there which I threw in to see if I could get it 
to work, and haven't yet had time to rewrite.

--
dwmw2

Re: DiskOnChip write performance

[Herman Oosthuysen]

In  my experience, a disk drive is always at least an order of magnitude slower
than the figure in the sales brochure, so the measured 70KB/s is probably
good...

Herman

On Thu, 06 Sep 2001, David Woodhouse wrote:
> herman@wirelessnetworksinc.com said:
> >  You may be able to speed the system up a little by optimizing the
> > code, but the improvement will be insignificant compared to the erase
> > cycle time.
> 
> Don't count on that if you're using my drivers. There are some fairly 
> inefficient algorithms in there which I threw in to see if I could get it 
> to work, and haven't yet had time to rewrite.
> 
> --
> dwmw2

Re: DiskOnChip write performance

[kira brown]

On Thu, 6 Sep 2001, Herman Oosthuysen wrote:

> You may be able to speed the system up a little by optimizing the code, but the
> improvement will be insignificant compared to the erase cycle time.

There are basically two ways to speed this situation up:

1)  Write to a RAM buffer and do the actual Flash write in the background.
(I'm sure you've thought of this).

2)  Make sure there are always freshly erased blocks on the Flash when you
come to need to do a fast write.

Either way, DOC2000 is never going to be any great shakes of speed-  it's
just not a very fast device.

kira.

Re: DiskOnChip write performance

[David Woodhouse]

kira@hex.linuxgrrls.org said:
>  There are basically two ways to speed this situation up:

Don't forget "don't use O(n^3) algorithms when trying to find a new block 
to write to".


--
dwmw2

Re: DiskOnChip write performance

[Ollie Lho]

David Woodhouse wrote:
> 
> kira@hex.linuxgrrls.org said:
> >  There are basically two ways to speed this situation up:
> 
> Don't forget "don't use O(n^3) algorithms when trying to find a new block
> to write to".
> 

What does this suppose to mean ??

Ollie

Re: DiskOnChip write performance

[Bjorn Eriksson]

On Fri, 7 Sep 2001, Ollie Lho wrote:

> > kira@hex.linuxgrrls.org said:
> > >  There are basically two ways to speed this situation up:
> > 
> > Don't forget "don't use O(n^3) algorithms when trying to find a new block
> > to write to".
> 
> What does this suppose to mean ??

 Read more on http://www.google.com/search?q=%22big+O+notation%22


-- 
//Björnen.

Re: DiskOnChip write performance

[David Woodhouse]

ollie@sis.com.tw said:
> > Don't forget "don't use O(n^3) algorithms when trying to find a new
> > block to write to".

> What does this suppose to mean ?? 

Algorithms which take time proportional to the cube of 'n', where n is some 
variable indicating the complexity of the task - in this case it would be 
the number of blocks on the DiskOnChip which are to be considered for 
selection. 

In fact, I don't think there's anything quite as stupid as O(n^3) in there, 
but it _is_ extremely naïve in places and needs a rethink of some of the 
data structures to allow some operations to operate in linear time, and to 
improve the wear levelling. The wear levelling at the moment is nonexistent 
- we always pick the longest chain, and if you have a one-block chain which 
never gets rewritten, NFTL will never cycle the block it's in.


> 	Did you "threaded" NFTL code yet ?? Or we still get locked when NFTL
> is syncing/flushing ?? 

I've put the necessary locking into the doc2000 driver. We can call it 
concurrently now. But haven't threaded NFTL to take advantage of it.

The core of the current NFTL code has just sort of evolved from my initial
hacks to see if I could get it to understand a DiskOnChip, and then to see
if I could write to it without confusing the DOS drivers. And it hasn't
evolved far. Ideally, it wants to be completely rewritten.


--
dwmw2

Re: DiskOnChip write performance

[Ollie Lho]

David Woodhouse wrote:
> 
> ollie@sis.com.tw said:
> > > Don't forget "don't use O(n^3) algorithms when trying to find a new
> > > block to write to".
> 
> > What does this suppose to mean ??
> 
> Algorithms which take time proportional to the cube of 'n', where n is some
> variable indicating the complexity of the task - in this case it would be
> the number of blocks on the DiskOnChip which are to be considered for
> selection.
> 

Yea, I know what Big O is although I am not CompSci graduate. My
question
is which part of the NFTL code is that bad ??

Ollie

Re: DiskOnChip write performance

[David Woodhouse]

ollie@sis.com.tw said:
>  Yea, I know what Big O is although I am not CompSci graduate. 

Sorry :)

> My question is which part of the NFTL code is that bad ??

The bit that selects a chain to fold in NFTL_makefreeblock is fairly crap, 
and the way we do findfreeblock(0)/makefreeblock()/findfreeblock(desperate)
in NFTL_findwriteunit() is also fairly crap.

That lot wants rethinking.



--
dwmw2

Re: DiskOnChip write performance

[David Woodhouse]

kira@hex.linuxgrrls.org said:
>  The M-Systems drivers are not the topic of this mailing list.

> (Sorry David, I had to say it.) 

Don't be sorry. You are entirely correct, and only saved me the hassle of 
saying it. Thank you.

We're interested in the behaviour of the M-Systems drivers only to the 
extent that we need to ensure interoperability and use them as a benchmark 
for our minimum acceptable performance.

--
dwmw2