Re: Opportunistic early flushing of data

[Hans Reiser <reiser@namesys.com>]

Xuan Baldauf wrote:

>Hans Reiser wrote:
>
>  
>
>>I just thought that discussion of this might interest the list.
>>
>>It is interesting to consider whether one should flush dirty nodes to
>>disk with only a small delay after they are modified, or keep them in
>>cache for a longer time.
>>
>>A small delay has an advantage for the typical small benchmark, and for
>>medium length tasks.  If there is poor utilization of the write cache,
>>then the sooner one gets started on flushing something to disk, the
>> more megabytes the disk can write before the benchmark ends.
>>
>>On the other hand, for a loaded server with a reasonably stable load
>>with real users not benchmarks, the longer data stays in cache the more
>>likely the write won't be needed at all.  One can argue that real users
>>don't create stable loads.....
>>
>>Your thoughts are welcome.
>>
>>--
>>Hans
>>    
>>
>
>What about opportunistic early flushing? If the disk is not accessed, we
>may flush without using someone elses resources. This cannot be implemented
>for all cases (because during the write, someone else may want to flush
>immediately), but there may be heuristics (like "we flush opportunistically
>if the disk has not been used for the last 100ms") which approximate to the
>optimal case. The more the disk is busy, the more flush should be delayed.
>
>This way, we use the advantage of early flush (early commit for database
>servers, etc.) with the advantages of late flush (less IO traffic). If
>there is low IO traffic, we would not need to lower it, we can do early
>flush. If there is high IO traffic, early flush would even rise it but late
>flush would lower it.
>
>It is important that the heuristics really depend on usage patterns of the
>disk, not only of the filesystem. There might be a system with a swap
>partition and a file system partition with heavy memory load and some disk
>load. The resulting heavy swap would be slowed down if the filesystem did
>early flush, and the filesystem would think that it is okay to do early
>flush due to the less IO throughput (which in turn is result of slowed
>swap...).
>
>Xuân.
>
>
>
>
>
>  
>
Xuan, I think you have the right approach here.  I think Daniel Phillips 
maybe outlined something somewhat similar some time ago on lkml, let's 
ask him.:)

-- 
Hans