any way to disable fsync?

any way to disable fsync?

[Łukasz Mierzwa]

Hi

I got problem with apps that are calling fsync, it makes my hard drive  
flush like mad and it slows down things quite a lot.
Example:
I'm using amarok and amrok uses sqlite for storing data, sqlite calls  
fsync after every write so on song change amarok writes to sqlite some  
statistic data about what I listen and sqllite calls fsync, now I got  
amarok trying to load next song and sqlite fsyncing and the problem is  
that fsyncing under reiser4 is expensive, it takes a lot of time even if  
I'm only running amarok and nothing else ( I got 4200RPM drive in my  
notebook so it;s not a speed daemon).  I patched sqlite in amarok to fake  
all fsync and it fixed it but there are other apps that try to secure my  
data by syncing (I'm now running azareus and only fsyncs called every  
minute can explain writing to disk at full speed almost non stop while I'm  
downloading only 64KB/s ).
So my question is: is there any way to disable fsync for reiser4? (beside  
patching it to fake fsync instead of doing them).
Thanks for great fs.

£ukasz Mierzwa

Re: any way to disable fsync?

[Jonathan Briggs]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

On Tue, 2006-07-11 at 23:03 +0200, Łukasz Mierzwa wrote:
[snip]
> So my question is: is there any way to disable fsync for reiser4? (beside  
> patching it to fake fsync instead of doing them).

http://ftp.die.net/pub/qmail-tools/libnosync.c

-- 
Jonathan Briggs <jbriggs@esoft.com>
eSoft, Inc.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: any way to disable fsync?

[Hans Reiser]

£ukasz Mierzwa wrote:

> Hi
>
> I got problem with apps that are calling fsync, it makes my hard
> drive  flush like mad and it slows down things quite a lot.
> Example:
> I'm using amarok and amrok uses sqlite for storing data, sqlite calls 
> fsync after every write so on song change amarok writes to sqlite
> some  statistic data about what I listen and sqllite calls fsync, now
> I got  amarok trying to load next song and sqlite fsyncing and the
> problem is  that fsyncing under reiser4 is expensive, it takes a lot
> of time even if  I'm only running amarok and nothing else ( I got
> 4200RPM drive in my  notebook so it;s not a speed daemon).  I patched
> sqlite in amarok to fake  all fsync and it fixed it but there are
> other apps that try to secure my  data by syncing (I'm now running
> azareus and only fsyncs called every  minute can explain writing to
> disk at full speed almost non stop while I'm  downloading only 64KB/s ).
> So my question is: is there any way to disable fsync for reiser4?
> (beside  patching it to fake fsync instead of doing them).
> Thanks for great fs.
>
> £ukasz Mierzwa
>
>
Just insert a return in the fsync code, it will be an amusing beginner
kernel project for you.  If you make it into a mount option we'll give
you credit in the code for it.

Alternatively, wait a few months, and fsync will be optimized.

Re: any way to disable fsync?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

On Tue, 11 Jul 2006 23:03:12 +0200, =?iso-8859-2?B?o3VrYXN6IE1pZXJ6d2E=?= said:
> I got problem with apps that are calling fsync, it makes my hard drive  
> flush like mad and it slows down things quite a lot.

Several have posted how to bypass it.  I'll pose the opposite side:

Usually, applications call fsync() because they're pretty sure that if
the disk and in-memory copies aren't lined up, a crash at that point could
result in data loss and/or corruption.

So sqlite calls fsync() - probably because if it *doesn't*, and your
system crashes/reboots, you *will* lose that sqlite database.

Your data, your decision.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: any way to disable fsync?

[Toby Thain]

On 11-Jul-06, at 5:57 PM, Valdis.Kletnieks@vt.edu wrote:

> On Tue, 11 Jul 2006 23:03:12 +0200, =?iso-8859-2?B? 
> o3VrYXN6IE1pZXJ6d2E=?= said:
>> I got problem with apps that are calling fsync, it makes my hard  
>> drive
>> flush like mad and it slows down things quite a lot.
>
> Several have posted how to bypass it.  I'll pose the opposite side:
>
> Usually, applications call fsync() because they're pretty sure that if
> the disk and in-memory copies aren't lined up, a crash at that  
> point could
> result in data loss and/or corruption.
>
> So sqlite calls fsync() - probably because if it *doesn't*, and your
> system crashes/reboots, you *will* lose that sqlite database.

Absolutely; it's required for commit semantics. :-)

>
> Your data, your decision.

Re: any way to disable fsync?

[Toby Thain]

On 11-Jul-06, at 5:57 PM, Valdis.Kletnieks@vt.edu wrote:

> On Tue, 11 Jul 2006 23:03:12 +0200, =?iso-8859-2?B? 
> o3VrYXN6IE1pZXJ6d2E=?= said:
>> I got problem with apps that are calling fsync, it makes my hard  
>> drive
>> flush like mad and it slows down things quite a lot.
>
> Several have posted how to bypass it.  I'll pose the opposite side:
>
> Usually, applications call fsync() because they're pretty sure that if
> the disk and in-memory copies aren't lined up, a crash at that  
> point could
> result in data loss and/or corruption.
>
> So sqlite calls fsync() - probably because if it *doesn't*, and your
> system crashes/reboots, you *will* lose that sqlite database.

Absolutely; it's required for commit semantics. :-)

>
> Your data, your decision.

Re: any way to disable fsync?

[Hans Reiser]

There are legitimate applications where the value of data is low enough
and the load is high enough, that losing the database upon crash is ok.

I have mixed feelings about making it a mount option for reiser4 because
many users will not know what they do.  In the end though, I should just
sell the rope and advise but not control what people do with it.   If
someone writes it I will take a mount option patch to disable fsync iff
it comes with documentation that has a lot of warnings.

Re: any way to disable fsync?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]

On Tue, 11 Jul 2006 17:04:56 PDT, Hans Reiser said:
> There are legitimate applications where the value of data is low enough
> and the load is high enough, that losing the database upon crash is ok.
>
> I have mixed feelings about making it a mount option for reiser4 because
> many users will not know what they do.  In the end though, I should just
> sell the rope and advise but not control what people do with it.   If
> someone writes it I will take a mount option patch to disable fsync iff
> it comes with documentation that has a lot of warnings.

Two things to consider before writing code:

1) Should it be done at the VFS level instead of in the filesystem?
Architecturally, it might be better there, so it applies to ext3 and jfs
and others too... 

2) Alternatively, should it be done on a per-file basis (possibly
flagged with a chattr or similar)?  It can't be done as an open()
flag or ioctl(), because you're trying to override what the code does...
That way, you can mitigate any fsync() load caused by one file, and
still not leave yourself open to being screwed by some other application
that tries to fsync() in other directories on that filesystem.  It
would be Really Bad if /home/fred/db23.sqlite gets corrupted because
the filesystem was mounted -nofsync because of /home/george/moby.sqlite
overhead....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: any way to disable fsync?

[Hans Reiser]

Valdis.Kletnieks@vt.edu wrote:

>On Tue, 11 Jul 2006 17:04:56 PDT, Hans Reiser said:
>  
>
>>There are legitimate applications where the value of data is low enough
>>and the load is high enough, that losing the database upon crash is ok.
>>
>>I have mixed feelings about making it a mount option for reiser4 because
>>many users will not know what they do.  In the end though, I should just
>>sell the rope and advise but not control what people do with it.   If
>>someone writes it I will take a mount option patch to disable fsync iff
>>it comes with documentation that has a lot of warnings.
>>    
>>
>
>Two things to consider before writing code:
>
>1) Should it be done at the VFS level instead of in the filesystem?
>Architecturally, it might be better there, so it applies to ext3 and jfs
>and others too...
>
Yes, if the below is not done.

> 
>
>2) Alternatively, should it be done on a per-file basis (possibly
>flagged with a chattr or similar)? 
>
Once the pseudofiles interface or sys_reiser4() are debugged and
working, yes.

> It can't be done as an open()
>flag or ioctl(), because you're trying to override what the code does...
>That way, you can mitigate any fsync() load caused by one file, and
>still not leave yourself open to being screwed by some other application
>that tries to fsync() in other directories on that filesystem.  It
>would be Really Bad if /home/fred/db23.sqlite gets corrupted because
>the filesystem was mounted -nofsync because of /home/george/moby.sqlite
>overhead....
>  
>

Re: any way to disable fsync?

[Łukasz Mierzwa]

Dnia Wed, 12 Jul 2006 01:41:10 +0200, Toby Thain <toby@smartgames.ca>  
napisa³:

>
> On 11-Jul-06, at 5:57 PM, Valdis.Kletnieks@vt.edu wrote:
>
>> On Tue, 11 Jul 2006 23:03:12 +0200, =?iso-8859-2?B? 
>> o3VrYXN6IE1pZXJ6d2E=?= said:
>>> I got problem with apps that are calling fsync, it makes my hard drive
>>> flush like mad and it slows down things quite a lot.
>>
>> Several have posted how to bypass it.  I'll pose the opposite side:
>>
>> Usually, applications call fsync() because they're pretty sure that if
>> the disk and in-memory copies aren't lined up, a crash at that point  
>> could
>> result in data loss and/or corruption.
>>
>> So sqlite calls fsync() - probably because if it *doesn't*, and your
>> system crashes/reboots, you *will* lose that sqlite database.
>
> Absolutely; it's required for commit semantics. :-)
>
>>
>> Your data, your decision.
>

I know it's there to keep data integrity but what should I do when it  
makes some apps unusable? Becouse it really hurts amarok for me and  
without disabling it in sqlite code amarok I can't call it listening to  
music. I can always patch one or few apps and live with it but what if  
half of my apps will do it?
Anyway I'm goint to work now so I won't be having this problem for next 8  
hours.