fsync performance of reiser 4

fsync performance of reiser 4

[Sandeep Tyagi]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

Hi -
Does anybody any idea whether reiserfs people are working on improvement of 
fsync performance of Reiser 4.
We have an application in which we use lot of fsyncs so we are planning to 
Reiser 4 if it gives good performance with fsync.
Thanks 

-- 
-----------
S K T
-----------

[-- Attachment #2: Type: text/html, Size: 369 bytes --]

Re: fsync performance of reiser 4

[Vladimir V. Saveliev]

Hello

Sandeep Tyagi wrote:
> Hi -
> Does anybody any idea whether reiserfs people are working on improvement
> of fsync performance of Reiser 4.

do you have any measurements already?

> We have an application in which we use lot of fsyncs so we are planning
> to Reiser 4 if it gives good performance with fsync.
> Thanks
> 
> -- 
> -----------
> S K T
> -----------

Re: fsync performance of reiser 4

[Ric Wheeler]

I have a file system benchmark that we use to measure synchronous file 
writing using different synch techniques.

I will be happy to share it if there is interest,

Ric


Vladimir V. Saveliev wrote:

>Hello
>
>Sandeep Tyagi wrote:
>  
>
>>Hi -
>>Does anybody any idea whether reiserfs people are working on improvement
>>of fsync performance of Reiser 4.
>>    
>>
>
>do you have any measurements already?
>
>  
>
>>We have an application in which we use lot of fsyncs so we are planning
>>to Reiser 4 if it gives good performance with fsync.
>>Thanks
>>
>>-- 
>>-----------
>>S K T
>>-----------
>>    
>>
>
>  
>

Re: fsync performance of reiser 4

[Grzegorz Kulewski]

On Mon, 29 Aug 2005, Ric Wheeler wrote:

>
> I have a file system benchmark that we use to measure synchronous file 
> writing using different synch techniques.
>
> I will be happy to share it if there is interest,

I am interested.


Thanks,

Grzegorz Kulewski

Re: fsync performance of reiser 4

[Ming Zhang]

On Mon, 2005-08-29 at 14:37 +0200, Grzegorz Kulewski wrote:
> On Mon, 29 Aug 2005, Ric Wheeler wrote:
> 
> >
> > I have a file system benchmark that we use to measure synchronous file 
> > writing using different synch techniques.
> >
> > I will be happy to share it if there is interest,
> 
> I am interested.
> 
> Thanks,
> 
> Grzegorz Kulewski


m2. will it be GPL'ed and put on SF.net or other similar sites?


Thanks!

ming

Re: fsync performance of reiser 4

[Ric Wheeler]

It is GPL'ed and has been posted at the project doubt website (which 
needs updating, http://developer.osdl.jp/projects/doubt/).  I will 
resend it directly to Ming and Grzegorz, but am not sure how best to 
share it with the whole list.

Ric

Ming Zhang wrote:

>On Mon, 2005-08-29 at 14:37 +0200, Grzegorz Kulewski wrote:
>  
>
>>On Mon, 29 Aug 2005, Ric Wheeler wrote:
>>
>>    
>>
>>>I have a file system benchmark that we use to measure synchronous file 
>>>writing using different synch techniques.
>>>
>>>I will be happy to share it if there is interest,
>>>      
>>>
>>I am interested.
>>
>>Thanks,
>>
>>Grzegorz Kulewski
>>    
>>
>
>
>m2. will it be GPL'ed and put on SF.net or other similar sites?
>
>
>Thanks!
>
>ming
>
>
>
>  
>

Re: fs_mark benchmark - update

[Ric Wheeler]

I use it to benchmark (mostly) synchronous file writing.

For example, a typical run might be to completely fill a file system 
with 50k files using the default (write one file, fsync one file) method 
and then compare that run to one which uses a varying number of 
concurrent threads.

Data journal mode is slightly faster for small files, but is twice as 
slow for large files (as expected).

We have observed the best results in 2.6 reiserfs3 when running 4-8 
threads per disk, using multiple subdirectories.  In 2.4, using a single 
subdirectory on reiser was better than using multiple subdirectories.

The validation of the write barrier case test is a simple one - if you 
write single threaded, then a properly configured file system with write 
barrier enabled should be some reasonable match to the speed of the 
disk.  For us, that is typically around 40 50k files/sec with S-ATA 
drives or P-ATA drives.

It is also interesting to compare reiserfs vs ext3 when writing single 
threaded across the life span of a file system - reiserfs typically 
beats ext3 for most cases, but ext3 eventually catches up as the 
percentage used grows.

I hope to rerun on reiserfs4 and the new ext3 in the next few weeks,

Regards,

Ric


Ming Zhang wrote:

>have a quick check on u site and it is interesting.
>
>but this is more like a validation tool instead of performance benchmark
>tool. or your code will report how many sync IOPS it gets and then we
>validate with our own HW spec.
>
>i should ask this after running the code i guess. :) but eager to know
>answer.
>
>ming
>
>
>
>  
>
>

Re: fs_mark benchmark - update

[Ming Zhang]

On Mon, 2005-08-29 at 09:46 -0400, Ric Wheeler wrote:
> I use it to benchmark (mostly) synchronous file writing.
> 
> For example, a typical run might be to completely fill a file system 
> with 50k files using the default (write one file, fsync one file) method 
> and then compare that run to one which uses a varying number of 
> concurrent threads.
> 
> Data journal mode is slightly faster for small files, but is twice as 
> slow for large files (as expected).
> 
> We have observed the best results in 2.6 reiserfs3 when running 4-8 
> threads per disk, using multiple subdirectories.  In 2.4, using a single 
> subdirectory on reiser was better than using multiple subdirectories.
> 
> The validation of the write barrier case test is a simple one - if you 
> write single threaded, then a properly configured file system with write 
> barrier enabled should be some reasonable match to the speed of the 
> disk.  For us, that is typically around 40 50k files/sec with S-ATA 
> drives or P-ATA drives.
> 
> It is also interesting to compare reiserfs vs ext3 when writing single 
> threaded across the life span of a file system - reiserfs typically 
> beats ext3 for most cases, but ext3 eventually catches up as the 
> percentage used grows.
> 

this is interesting. looks like a file system aging problem. does this
because of the fragmentation?

> I hope to rerun on reiserfs4 and the new ext3 in the next few weeks,
> 

looking forward to seeing.

what u mean new ext3?

thanks!

ming


> Regards,
> 
> Ric
> 
> 
> Ming Zhang wrote:
> 
> >have a quick check on u site and it is interesting.
> >
> >but this is more like a validation tool instead of performance benchmark
> >tool. or your code will report how many sync IOPS it gets and then we
> >validate with our own HW spec.
> >
> >i should ask this after running the code i guess. :) but eager to know
> >answer.
> >
> >ming
> >
> >
> >
> >  
> >
> >

Re: fs_mark benchmark - update

[Ric Wheeler]

Ming Zhang wrote:

>On Mon, 2005-08-29 at 09:46 -0400, Ric Wheeler wrote:
>  
>
>>I use it to benchmark (mostly) synchronous file writing.
>>
>>For example, a typical run might be to completely fill a file system 
>>with 50k files using the default (write one file, fsync one file) method 
>>and then compare that run to one which uses a varying number of 
>>concurrent threads.
>>
>>Data journal mode is slightly faster for small files, but is twice as 
>>slow for large files (as expected).
>>
>>We have observed the best results in 2.6 reiserfs3 when running 4-8 
>>threads per disk, using multiple subdirectories.  In 2.4, using a single 
>>subdirectory on reiser was better than using multiple subdirectories.
>>
>>The validation of the write barrier case test is a simple one - if you 
>>write single threaded, then a properly configured file system with write 
>>barrier enabled should be some reasonable match to the speed of the 
>>disk.  For us, that is typically around 40 50k files/sec with S-ATA 
>>drives or P-ATA drives.
>>
>>It is also interesting to compare reiserfs vs ext3 when writing single 
>>threaded across the life span of a file system - reiserfs typically 
>>beats ext3 for most cases, but ext3 eventually catches up as the 
>>percentage used grows.
>>
>>    
>>
>
>this is interesting. looks like a file system aging problem. does this
>because of the fragmentation?
>  
>

I think that this is because of the depth and size of the tree.  We have 
very large volumes (up to 300GB) - if your file systems are smaller, 
then the degradation is not as large.

>  
>
>>I hope to rerun on reiserfs4 and the new ext3 in the next few weeks,
>>
>>    
>>
>
>looking forward to seeing.
>
>what u mean new ext3?
>  
>
I should have said "new patches to ext3".  At OLS, there was a talk 
describing this work - you can get the proceedings from linuxsymposium.org:

http://www.linuxsymposium.org/2005/view_abstract.php?content_key=90


>thanks!
>
>ming
>  
>

Re: fs_mark benchmark - update

[Ming Zhang]

On Mon, 2005-08-29 at 10:07 -0400, Ric Wheeler wrote:
> 
> Ming Zhang wrote:
> 
> >On Mon, 2005-08-29 at 09:46 -0400, Ric Wheeler wrote:
> >  
> >>It is also interesting to compare reiserfs vs ext3 when writing single 
> >>threaded across the life span of a file system - reiserfs typically 
> >>beats ext3 for most cases, but ext3 eventually catches up as the 
> >>percentage used grows.
> >>
> >>    
> >>
> >
> >this is interesting. looks like a file system aging problem. does this
> >because of the fragmentation?
> >  
> >
> 
> I think that this is because of the depth and size of the tree.  We have 
> very large volumes (up to 300GB) - if your file systems are smaller, 
> then the degradation is not as large.
> 

so will this be a scalability problem? all because we have 4KB block
size instead of using 64KB block size and pack small data using that
"tail" option?

500GB SATA is easy to buy and TB storage is not a dream at all. so this
will be a problem.


> >
> >what u mean new ext3?
> >  
> >
> I should have said "new patches to ext3".  At OLS, there was a talk 
> describing this work - you can get the proceedings from linuxsymposium.org:
> 
> http://www.linuxsymposium.org/2005/view_abstract.php?content_key=90
> 
> 

ic. thanks!

ming

Re: fs_mark benchmark - update

[Hans Reiser]

reiser4 will suck at fsync performance, I guarantee it.  fsync is
completely unoptimized.  It works reliably, but it will be slow. 

The suckage is all eminently fixable,  but not before vanilla inclusion
is resolved will we address it.

Re: fs_mark benchmark - update

[Ric Wheeler]

Since you still are in the thinking stage, there are some neat tricks 
that you might be able to do in this space ;-)

One thing that I think might be very interesting is to export some of 
the transaction like semantics explicitely to applications to allow a 
type of bulk async fsync. For example, I am untarring a 10,000 file set 
into a directory - you want a hard promise that it is all on disk, but 
you really don't need that promise until the end of the application.  If 
the application could flag the start of this kind of batch and then wait 
or kick off a group fsync at the end, you could really boost your 
performance.

The benchmark has some kludgy support for testing this kind of sync 
performance through the "-S X" flag.  When X==0, you don't sync, when it 
is 1 you do the file at a time fsync, and then it does some other 
combinations that work well on reiserfs.  By simply delaying the fsync() 
stage into a separate iteration over the file set, you can approach the 
non-fsynced behavior ;-)

ric


Hans Reiser wrote:

>reiser4 will suck at fsync performance, I guarantee it.  fsync is
>completely unoptimized.  It works reliably, but it will be slow. 
>
>The suckage is all eminently fixable,  but not before vanilla inclusion
>is resolved will we address it.
>  
>

Re: fs_mark benchmark - update

[Hans Reiser]

Ric Wheeler wrote:

>
> Since you still are in the thinking stage, there are some neat tricks
> that you might be able to do in this space ;-)
>
> One thing that I think might be very interesting is to export some of
> the transaction like semantics explicitely to applications to allow a
> type of bulk async fsync. For example, I am untarring a 10,000 file
> set into a directory - you want a hard promise that it is all on disk,
> but you really don't need that promise until the end of the
> application.  If the application could flag the start of this kind of
> batch and then wait or kick off a group fsync at the end, you could
> really boost your performance.

Yes, that was our idea.  It needs just a little bit of API coding....
and it will be there..... but of course we are all focused on getting
ready for the merge.

>
> The benchmark has some kludgy support for testing this kind of sync
> performance through the "-S X" flag.  When X==0, you don't sync, when
> it is 1 you do the file at a time fsync, and then it does some other
> combinations that work well on reiserfs.  By simply delaying the
> fsync() stage into a separate iteration over the file set, you can
> approach the non-fsynced behavior ;-)
>
> ric
>
>
> Hans Reiser wrote:
>
>> reiser4 will suck at fsync performance, I guarantee it.  fsync is
>> completely unoptimized.  It works reliably, but it will be slow.
>> The suckage is all eminently fixable,  but not before vanilla inclusion
>> is resolved will we address it.
>>  
>>
>
>