Feature request: online fsync disable

Feature request: online fsync disable

[Xuan Baldauf]

Hello,

I have a server running which sometimes gets very busy. The
running application issues lots of fsync() calls. If it gets
very busy, it sometimes is not as fast as required. I
believe that most of the time is spent in processing fsync()
(commit journal, etc.). In such a situation, I'd like to
switch off all fsync()s and risk data loss instead of hours
of processing time, which in my case is more important.

Is implementing such a feature possible? It may be used as
follows:

mount /partition -o remount,disableFsync

and undone as follows:

mount /partition -o remount,enableFsync


Xuân.

Re: Feature request: online fsync disable

[Nikita Danilov]

Xuan Baldauf writes:
 > Hello,
 > 
 > I have a server running which sometimes gets very busy. The
 > running application issues lots of fsync() calls. If it gets
 > very busy, it sometimes is not as fast as required. I
 > believe that most of the time is spent in processing fsync()
 > (commit journal, etc.). In such a situation, I'd like to
 > switch off all fsync()s and risk data loss instead of hours
 > of processing time, which in my case is more important.
 > 
 > Is implementing such a feature possible? It may be used as
 > follows:
 > 
 > mount /partition -o remount,disableFsync
 > 
 > and undone as follows:
 > 
 > mount /partition -o remount,enableFsync
 > 

I don't know your particulars, but one way to do such thing is to write
small shared library implementing no-op fsync() function and to use
LD_PRELOAD.

 > 
 > Xuân.
 > 

Nikita.

 > 

Re: Feature request: online fsync disable

[Oleg Drokin]

Hello!

On Fri, Aug 23, 2002 at 03:52:38PM +0400, Nikita Danilov wrote:
>  > I have a server running which sometimes gets very busy. The
>  > running application issues lots of fsync() calls. If it gets
>  > very busy, it sometimes is not as fast as required. I
>  > believe that most of the time is spent in processing fsync()
>  > (commit journal, etc.). In such a situation, I'd like to
>  > switch off all fsync()s and risk data loss instead of hours
>  > of processing time, which in my case is more important.
>  > 
>  > Is implementing such a feature possible? It may be used as
>  > follows:
>  > 
>  > mount /partition -o remount,disableFsync
>  > 
>  > and undone as follows:
>  > 
>  > mount /partition -o remount,enableFsync
> I don't know your particulars, but one way to do such thing is to write
> small shared library implementing no-op fsync() function and to use
> LD_PRELOAD.

Also there is http://namesys.com/support.html page that describes how one
can get NAMESYS to write virtually any reiserfs feature ;)

Bye,
    Oleg

Re: Feature request: online fsync disable

[Hans Reiser]

It is a reasonable feature request I think.

Zam, put it into reiser4.1.

Re: Feature request: online fsync disable

[Nikita Danilov]

Hans Reiser writes:
 > It is a reasonable feature request I think.
 > 
 > Zam, put it into reiser4.1.
 > 

I think, in stead of feature "disable fsync", better alternative would
be "disable journalling", or "journal meta-data only".

Nikita.

Re: Feature request: online fsync disable

[Chris Mason]

On Fri, 2002-08-23 at 08:09, Nikita Danilov wrote:
> Hans Reiser writes:
>  > It is a reasonable feature request I think.
>  > 
>  > Zam, put it into reiser4.1.
>  > 
> 
> I think, in stead of feature "disable fsync", better alternative would
> be "disable journalling", or "journal meta-data only".

Well, if the application doesn't really need fsync, it shouldn't call
it.  I'd much rather fix whatever problem is making things too slow.

Xuan, you could try the data logging patches in either data=writeback or
data=journal.  Either one will be significantly faster than stock
2.4.19.

-chris

Feature request: online fsync disable

[Xuan Baldauf]

Hello,

I have a server running which sometimes gets very busy. The
running application issues lots of fsync() calls. If it gets
very busy, it sometimes is not as fast as required. I
believe that most of the time is spent in processing fsync()
(commit journal, etc.). In such a situation, I'd like to
switch off all fsync()s and risk data loss instead of hours
of processing time, which in my case is more important.

Is implementing such a feature possible? It may be used as
follows:

mount /partition -o remount,disableFsync

and undone as follows:

mount /partition -o remount,enableFsync


Xuân.

Re: Feature request: online fsync disable

[Xuan Baldauf]

Chris Mason wrote:

> On Fri, 2002-08-23 at 08:09, Nikita Danilov wrote:
> > Hans Reiser writes:
> >  > It is a reasonable feature request I think.
> >  >
> >  > Zam, put it into reiser4.1.
> >  >
> >
> > I think, in stead of feature "disable fsync", better alternative would
> > be "disable journalling", or "journal meta-data only".
>
> Well, if the application doesn't really need fsync, it shouldn't call
> it.  I'd much rather fix whatever problem is making things too slow.

It's like that: Crashing with data loss creates trouble, but that trouble is
fixable. In our case, being slower than required for some time creates much
more trouble.

>
>
> Xuan, you could try the data logging patches in either data=writeback or
> data=journal.  Either one will be significantly faster than stock
> 2.4.19.

Thank you. I guess "data=writeback" will result in the desired behaviour.

Are these patches tested?

>
>
> -chris

Re: Feature request: online fsync disable

[Chris Mason]

On Fri, 2002-08-23 at 14:10, Xuan Baldauf wrote:

> > Xuan, you could try the data logging patches in either data=writeback or
> > data=journal.  Either one will be significantly faster than stock
> > 2.4.19.
> 
> Thank you. I guess "data=writeback" will result in the desired behaviour.
> 
> Are these patches tested?

Yes, although I've now got 3 or 4 bug fixes from testing in the suse
kernel that I need to put into the pure 2.4.19 patch.  I'm doing the
last of the fixes today, and I'll start merging back into the main code
on monday.  This was all supposed to happen last week, but I got
diverted by some important other suse work.

If your application is very fsync heavy and doesn't do many non-fsync
writes data=journal will probably be much faster.

-chris

Re: Feature request: online fsync disable

[Matthias Andree]

Hans Reiser <reiser@namesys.com> writes:

> It is a reasonable feature request I think.

A file system that has a switch "oh, you can throw away my data if you
like" is no good at all. If the application + FS interaction leads to
contention, then you have to choices to optimize: the application or the
file system. We've had this kind of discussion before, with a different
focus: on mail transfer agents. There have been considerations of
merging fsync() operations and committing them at the same time, and
asynchronous operation or threading might in fact help the application.

I'd not trust a file system that even has a switch to let it ignore
fsync(). Please let me know if your file systems exhibit this behaviour
so I can statfs() in my applications and if I see a file system that
does ignore fsync() just call raise(SIGKILL);

-- 
Matthias Andree

Re: Feature request: online fsync disable

[Matthias Andree]

Xuan Baldauf <xuan--reiserfs@baldauf.org> writes:

> It's like that: Crashing with data loss creates trouble, but that trouble is
> fixable. In our case, being slower than required for some time creates much
> more trouble.

So don't call fsync(), or call it less frequently.

-- 
Matthias Andree

Re: Feature request: online fsync disable

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

On Sun, 25 Aug 2002 23:21:35 +0200, Matthias Andree <ma+rfs@dt.e-technik.uni-dortmund.de>  said:

> So don't call fsync(), or call it less frequently.

This is, of course, assuming that you have any actual control over the
matter.  Very few sites actually control their low-level database code,
as it's something that's fiendishly difficult to do well.  As a result,
you have two major trends:

1) You run Oracle or Sleepycat or some SQL product, and don't have much
control over when the fsync's get done (or even worse, you're using some
fourth-party package that *itself* calls Oracle or Sleepycat or...).

2) Said products (at least the good ones) have probably already had most
of the truly egregious abuses of fsync() optimized out, so there isn't
much win to trying to optimize further...

(And yes, I *know* Oracle can be configured to use raw disk partitions.
That doesn't change the underlying logic above) 
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Feature request: online fsync disable

[Hans Reiser]

Matthias Andree wrote:

>Hans Reiser <reiser@namesys.com> writes:
>
>  
>
>>It is a reasonable feature request I think.
>>    
>>
>
>A file system that has a switch "oh, you can throw away my data if you
>like" is no good at all.
>
Yes it is, consider tmpfs, consider web cache appliances.....  this is 
just another variation on it.

> If the application + FS interaction leads to
>contention, then you have to choices to optimize: the application or the
>file system.
>
If the app is closed source....

> We've had this kind of discussion before, with a different
>focus: on mail transfer agents. There have been considerations of
>merging fsync() operations and committing them at the same time, and
>asynchronous operation or threading might in fact help the application.
>
>I'd not trust a file system that even has a switch to let it ignore
>fsync(). Please let me know if your file systems exhibit this behaviour
>so I can statfs() in my applications and if I see a file system that
>does ignore fsync() just call raise(SIGKILL);
>
>  
>

Re: Feature request: online fsync disable

[Xuan Baldauf]

Chris Mason wrote:

> On Fri, 2002-08-23 at 14:10, Xuan Baldauf wrote:
>
> > > Xuan, you could try the data logging patches in either data=writeback or
> > > data=journal.  Either one will be significantly faster than stock
> > > 2.4.19.
> >
> > Thank you. I guess "data=writeback" will result in the desired behaviour.
> >
> > Are these patches tested?
>
> Yes, although I've now got 3 or 4 bug fixes from testing in the suse
> kernel that I need to put into the pure 2.4.19 patch.  I'm doing the
> last of the fixes today, and I'll start merging back into the main code
> on monday.  This was all supposed to happen last week, but I got
> diverted by some important other suse work.
>
> If your application is very fsync heavy and doesn't do many non-fsync
> writes data=journal will probably be much faster.
>
> -chris

Hello Chris,

I've been searching for the loggin patches at ftp://ftp.namesys.com/ and below,
but could not find any patch named "logging" or containg the word "writeback".
Can you point me to the logging patches?

Thank you. :-)

Xuân.

Re: Feature request: online fsync disable

[Oleg Drokin]

Hello!

On Wed, Sep 04, 2002 at 01:39:30PM +0200, Xuan Baldauf wrote:
 
> I've been searching for the loggin patches at ftp://ftp.namesys.com/ and below,
> but could not find any patch named "logging" or containg the word "writeback".
> Can you point me to the logging patches?

ftp.suse.com:/pub/people/mason/patches/data-logging

Bye,
    Oleg

Re: Feature request: online fsync disable

[Hans Reiser]

Chris Wedgwood wrote:

>On Fri, Aug 23, 2002 at 03:59:58PM +0400, Hans Reiser wrote:
>
>    It is a reasonable feature request I think.
>
>No it's not.
>
>It's a terrible idea.  If someone calls fsync, they call it for a
>reason and have various expectations.
>
Usually these expectations involve a user not wanting to use mkreiserfs 
at every boot.  If the expectations are wrong, then all power to the user.

>  You should not throw these
>away or else you may as well just start writing random data all over
>the place.
>
>
>
>  --cw
>
>
>  
>