reiserfs performance on ssd

reiserfs performance on ssd

[Jure Pečar]

Hi all,

I have a simple solid state disk to play with here. 
See http://nerv.eu.org/iram/

Reiserfs is known to have good performance with small files. I am a big fan of it and am using it in production, however the numbers I got here got me thinking. fsstone is basically a create-rename-delete test ... is my conlcusion right that tree based disk organisation takes its toll here and makes it look 10 times slower than ext2?

-- 

Jure Pečar
http://jure.pecar.org/

Re: reiserfs performance on ssd

[Sander]

Jure Pe??ar wrote (ao):
> I have a simple solid state disk to play with here. 
> See http://nerv.eu.org/iram/

Interesting review, thanks.

To get better reliability you could raid1 them.
I guess this is a 'must' anyway when used in servers (just like with
harddisks).

Have to try this product myself..

	Kind regards, Sander

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: reiserfs performance on ssd

[Gregory Maxwell]

On 4/27/06, Sander <sander@humilis.net> wrote:
> > I have a simple solid state disk to play with here.
> > See http://nerv.eu.org/iram/
>
> Interesting review, thanks.
>
> To get better reliability you could raid1 them.
> I guess this is a 'must' anyway when used in servers (just like with
> harddisks).
>
> Have to try this product myself..

Because they have no ECC most failures will just be completely silent
data corruption.
A sadly useless device.

Re: reiserfs performance on ssd

[Toby Thain]

On 27-Apr-06, at 10:28 AM, Gregory Maxwell wrote:

> On 4/27/06, Sander <sander@humilis.net> wrote:
>>> I have a simple solid state disk to play with here.
>>> See http://nerv.eu.org/iram/
>>
>> Interesting review, thanks.
>>
>> To get better reliability you could raid1 them.
>> I guess this is a 'must' anyway when used in servers (just like with
>> harddisks).
>>
>> Have to try this product myself..
>
> Because they have no ECC most failures will just be completely silent
> data corruption.
> A sadly useless device.

Sure ECC would be nice, but how does this differ from disk? Silent  
failures are certainly possible.

The fact that error detection and propagation doesn't really happen  
in modern disk subsystems is why systems like Sun's ZFS are coming  
into being.

--T

Re: reiserfs performance on ssd

[Toby Thain]

On 27-Apr-06, at 10:28 AM, Gregory Maxwell wrote:

> On 4/27/06, Sander <sander@humilis.net> wrote:
>>> I have a simple solid state disk to play with here.
>>> See http://nerv.eu.org/iram/
>>
>> Interesting review, thanks.
>>
>> To get better reliability you could raid1 them.
>> I guess this is a 'must' anyway when used in servers (just like with
>> harddisks).
>>
>> Have to try this product myself..
>
> Because they have no ECC most failures will just be completely silent
> data corruption.
> A sadly useless device.

Sure ECC would be nice, but how does this differ from disk? Silent  
failures are certainly possible.

The fact that error detection and propagation doesn't really happen  
in modern disk subsystems is why systems like Sun's ZFS are coming  
into being.

--T

Re: reiserfs performance on ssd

[Gregory Maxwell]

On 4/27/06, Toby Thain <toby@smartgames.ca> wrote:
> Sure ECC would be nice, but how does this differ from disk? Silent
> failures are certainly possible.
>
> The fact that error detection and propagation doesn't really happen
> in modern disk subsystems is why systems like Sun's ZFS are coming
> into being.

Um. Because *every* cosmic ray hit (of which you can expect one to two
every week or so with 2+ gigs of ram) will result in data corruption.

It's claimed that disks don't do a great job propagating hard errors,
which is true to an extent. But they *do* manage to handle soft
errors. Without the coding gain provided by block ECC your modern high
density would be nearly useless.

ZFS wouldn't make iram seriously useable... because AFAIK, raidz will
not work on a single device... so even if can detect a bad block, it
can't correct it.

The problem goes further than that because the cost of computing block
checksums in software will greatly reduce the performance of the fast
ram device..

Not that better integrity features are bad.. The iron filesystem paper
has a lot of great suggestions that go beyond what ZFS provides, and
it would be wonderful to see them in reiser4 someday.  But things need
to progress one step at a time.

Re: reiserfs performance on ssd

[PFC]

	While I like the idea, the iram implementation is horrible for various  
reasons :

	- no ECC
	- It uses SATA hence only a very little part of the RAM speed is used,  
and large latencies are introduced.
	- I wouldn't trust it for critical data.

	Then, it would be faster to just add the RAM on the motherboard and  
create a ramdisk...

	Also : THE PCI BUS MUST DIE !

	It's no wonder reiserfs is slower on this thing ; reiserfs is optimized  
for hard disks which have high seek latencies.
	For RAM storage, things like hash tables are a lot better than trees,  
because you don't care about sequential scanning and locality of  
reference... (as long as the items are a bit larger than a cache line,  
which is obviously the case here).
	A filesystem that pauses and thinks for 0.5 ms of CPU time to avoid a 5  
ms HDD seek time is a good thing, but for SSD storage you want a FS that  
doesn't think at all.
	Using ReiserFS on this is like hitting a nail with a screwdriver.

	Try tmpfs, I'm curious about the results.

	Crazy idea :
	- Ask the motherboard manufacturer to stick a battery on the mobo so that  
the contents of RAM survive reboot
	- Tweak the BIOS so that RAM isn't cleared at boot
	- Teach the Kernel to be cautious while booting and not overwrite the  
previous RAMDISK
	- You have persistent ramdisk. Stick your reiserfs journal on it, enjoy.

Re: reiserfs performance on ssd

[Andreas Dilger]

On Apr 28, 2006  09:07 +0200, PFC wrote:
> 	While I like the idea, the iram implementation is horrible for 
> 	various  reasons :
> 
> 	- no ECC

I don't know why people are so keen on ECC RAM.  Why not just put an extra
socket on the board and run the RAM in "RAIM" (RAID for Memory) mode?
The incremental cost of ECC vs. regular RAM is FAR more than the cost
of just getting the extra stick of RAM.  Also, with RAIM you could even
hot-swap a failing DIMM, while with ECC you have to take an outage to
get back to redundancy.

> 	- It uses SATA hence only a very little part of the RAM speed is 
> 	used,  and large latencies are introduced.

Ah, but if you can connect the RAM to multiple machines, you could at
least have the hope of hot failover for the storage to another server.
That isn't something you can do with a bus-attached device.

Even the chance of any recovery is far better with such a setup (i.e.
plug into another system after motherboard dies), since you can at
least get access from another machine.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: reiserfs performance on ssd

[Jure Pečar]

On Fri, 28 Apr 2006 02:20:02 -0600
Andreas Dilger <adilger@clusterfs.com> wrote:

> I don't know why people are so keen on ECC RAM.  Why not just put an
> extra socket on the board and run the RAM in "RAIM" (RAID for Memory)
> mode? The incremental cost of ECC vs. regular RAM is FAR more than
> the cost of just getting the extra stick of RAM.  Also, with RAIM you
> could even hot-swap a failing DIMM, while with ECC you have to take
> an outage to get back to redundancy.

If I remember correctly, old Alphas used this method - 5 slots per memory bank.

> Ah, but if you can connect the RAM to multiple machines, you could at
> least have the hope of hot failover for the storage to another server.
> That isn't something you can do with a bus-attached device.

SATA doesn't allow you to connect a device to multiple machines (at least not yet, who knows what SAS will bring in the future). If you want to have SSD with ability to hang multiple machines on it, you have to look for fibre channel models which are waay to expensive.
 
> Even the chance of any recovery is far better with such a setup (i.e.
> plug into another system after motherboard dies), since you can at
> least get access from another machine.

Another example of "ram with battery" are umem cards, which are very difficult to obtain. But they do work at pci bus speeds. Linux supportd 5415 and 5425 models. I managed to get a 5420 model, which looks like it was made specifficaly for Sun and doesn't work with the existing driver. I'm hacking it in my spare time and I hope I can get it working...

-- 

Jure Pečar
http://jure.pecar.org

Re: reiserfs performance on ssd

[Perry Kundert]

On 4/28/06, Andreas Dilger <adilger@clusterfs.com> wrote:
> On Apr 28, 2006  09:07 +0200, PFC wrote:
> >       While I like the idea, the iram implementation is horrible for
> >       various  reasons :
> >
> >       - no ECC
>
> I don't know why people are so keen on ECC RAM.  Why not just put an extra
> socket on the board and run the RAM in "RAIM" (RAID for Memory) mode?
> The incremental cost of ECC vs. regular RAM is FAR more than the cost
> of just getting the extra stick of RAM.  Also, with RAIM you could even
> hot-swap a failing DIMM, while with ECC you have to take an outage to
> get back to redundancy.
>
    RAID on disks only detects errors when the underlying device
reports them.  Things like corruption "on the wire" (eg. bad cables)
are NOT detected.

    Similarly, no matter how many redundant sticks of RAM you have,
you MUST HAVE some hardware, somewhere, doing some computation on all
the bits flowing by -- even something as simple as a "best 2 of 3"
comparison -- to enable you to detect bad bits, and lock out and/or
refresh the bad DIMM.



--
    -pjk

Re: reiserfs performance on ssd

[Sander]

Ming Zhang wrote (ao):
> read u review. one thing i am not quite understand is the
> 
> "Numbers speak from themselves. It's interesting to note that all
> filesystems top the write speed at ~106MB/s and read speed at
> ~125MB/s.  It seem we're hitting 32bit PCI throughput limit here ."
> 
> since u said the iram use pci slot for power purpose. then where is
> this pci slot limit coming from?

The sata controller is on pci. Btw, pci is not a slot per se.

	Kind regards, Sander

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: reiserfs performance on ssd

[Ming Zhang]

o, yes. thx.


http://www.anandtech.com/storage/showdoc.aspx?i=2480&p=10

see this review, when copy a large iso file, the speed is still around
~100MB/s. and this review also use a ASUS A8N motherboard.

so none of these tests can 100% reflect the real performance. we need a
nice SATA controller and MB here.

Ming


On Thu, 2006-04-27 at 15:48 +0200, Sander wrote:
> Ming Zhang wrote (ao):
> > read u review. one thing i am not quite understand is the
> > 
> > "Numbers speak from themselves. It's interesting to note that all
> > filesystems top the write speed at ~106MB/s and read speed at
> > ~125MB/s.  It seem we're hitting 32bit PCI throughput limit here ."
> > 
> > since u said the iram use pci slot for power purpose. then where is
> > this pci slot limit coming from?
> 
> The sata controller is on pci. Btw, pci is not a slot per se.
> 
> 	Kind regards, Sander
>