Atomicity with reiser4?

Atomicity with reiser4?

[Nicolas Smallwood]

[-- Attachment #1.1: Type: text/plain, Size: 766 bytes --]

Network BlitzHello,

I was running a test with some in-house database software that relies on
fsync after transactions are complete.  I noticed severe perfomance lapses,
and have bee profiling the software for several days.   A minute ago, I
removed the fsync from the system, only making it think it had been
fsyncing.  This reduced my test from 15 minutes to less than 40 seconds.

Obviously there is a performance hit with using fsync, but we are of course
worried about database integrity.

Is there an avaliable paper on how reiser4 handles atomicity, or can it be
easily explained?
Essentially, my question is whether our database (using 4k blocksizes) will
lose data during power loss or other occurrences if we do not fsync?

Thank You,

Nicolas Smallwood



[-- Attachment #1.2: Type: text/html, Size: 1957 bytes --]

Re: Atomicity with reiser4?

[Vladimir Saveliev]

Hello

On Wed, 2005-02-02 at 22:36, Nicolas Smallwood wrote:
> Hello,
>  
> I was running a test with some in-house database software that relies
> on fsync after transactions are complete.  I noticed severe perfomance
> lapses, and have bee profiling the software for several days.   A
> minute ago, I removed the fsync from the system, only making it think
> it had been fsyncing.  This reduced my test from 15 minutes to less
> than 40 seconds.
>  

How often do you call fsync?
If you used reiserfs or ext2 - how much would your software performance
change when fsync is removed?
Would you brifly describe what kind of load for filesystem does your
software generate?

> Obviously there is a performance hit with using fsync, but we are of
> course worried about database integrity.
>  
> Is there an avaliable paper on how reiser4 handles atomicity, or can
> it be easily explained?

http://namesys.com/v4/v4.html#atomic_fs

> Essentially, my question is whether our database (using 4k blocksizes)
> will lose data during power loss or other occurrences if we do not
> fsync?
>  

Reiser4 flushes data to disk by portions of modified data called atom.
Atom may contain modifed data of several system calls. Flushing one atom
is called atom commit.
Atom gets committed whenever it becames too old or too large. They can
be forced to commit by fsync. 

Data of all atoms which were not committed at the moment of power loss
will be lost.

> Thank You,
>  
> Nicolas Smallwood
> 
>  
> 

Re: Atomicity with reiser4?

[Hans Reiser]

I think it is essential that you use the atomic transactions 
functionality to get good performance on top of reiser4 for what you are 
doing.  Unfortunately, that API is still kernel internal at this time, 
and we have not yet coded the export of it to outside the kernel. We 
will though, we just need some time.

Hans

RE: Atomicity with reiser4?

[Nicolas Smallwood]

Its an internally built B Tree database that uses 4k blocksizes.
Inside, we were loading "u-forms", a concept we use that involves
having a UUID, and an e-form composed off attribute names and their values.
Most of these are no more than say 17k.

To ensure that writes do occur, we open transactions for every access to the
b tree,
and closing a transaction will call fsync.

My test involved 20,000 u-forms, which would of course call fsync 20,000
times.

Using another of the filesystems, each is under or around two minutes with
fsync.

Based upon what you where telling me, is the size of an atom configurable?
Or what is considered too large?

Thanks,

Nicolas Smallwood

Re: Atomicity with reiser4?

[Hans Reiser]

Nicolas Smallwood wrote:

>Its an internally built B Tree database that uses 4k blocksizes.
>Inside, we were loading "u-forms", a concept we use that involves
>having a UUID, and an e-form composed off attribute names and their values.
>Most of these are no more than say 17k.
>
>To ensure that writes do occur, we open transactions for every access to the
>b tree,
>and closing a transaction will call fsync.
>
>My test involved 20,000 u-forms, which would of course call fsync 20,000
>times.
>
>Using another of the filesystems, each is under or around two minutes with
>fsync.
>
>Based upon what you where telling me, is the size of an atom configurable?
>Or what is considered too large?
>
>Thanks,
>
>Nicolas Smallwood
>
>
>
>
>  
>
You might consider creating u-form item plugins and doing it within 
reiser4.....  performance should be drastically improved by that.....