Re: dirty pages (Was: Re: [PATCH] Prevent large file writeback starvation)

[Alexander Bergolth <leo@strike.wu-wien.ac.at>]

On 02/13/2006 02:59 PM, Johannes Stezenbach wrote:
> On Mon, Feb 06, 2006, Andrew Morton wrote:
>>Mark Lord <lkml@rtr.ca> wrote:
>>
>>>A simple test I do for this:
>>>
>>> $ mkdir t
>>> $ cp /usr/src/*.bz2  t    (about 400-500MB worth of kernel tar files)
>>>
>>> In another window, I do this:
>>>
>>> $ while (sleep 1); do echo -n "`date`: "; grep Dirty /proc/meminfo; done
>>>
>>> And then watch the count get large, but take virtually forever
>>> to count back down to a "safe" value.
>>>
>>> Typing "sync" causes all the Dirty pages to immediately be flushed to disk,
>>> as expected.
>>
>>I've never seen that happen and I don't recall seeing any other reports of
>>it, so your machine must be doing something peculiar.  I think it can
>>happen if, say, an inode gets itself onto the wrong inode list, or
>>incorrectly gets its dirty flag cleared.

Are you still suffering from this problem?
I'm seeing exactly the same issue on many boxes (including
kernel-2.6.16). (See my tests and the corresponding test-script at [1].)

Are you using Software-Suspend? I am able to trigger the problem after
suspending to disk and resuming (swsuspend2). However, it does also
happen on some boxes with a stock fedora kernel and without suspending,
though it is hard to reproduce on those boxes. (But once it is
triggered, it doesn't go away anymore.)

On most machines that show the problem, buffers are flushed only based
on the amount of dirty memory (dirty_ratio and dirty_background_ratio)
or when doing a "sync", they are not periodically flushed anymore.
There is one box where even "sync" does only flush a fraction of the
dirty buffers [2].

I am not using vmware, no laptop_mode and no custom /proc/sys/vm/dirty*
settings are involved.

Cheers,
--leo

[1] Tests results and test-script:
http://leo.kloburg.at/tmp/dirty-buffers/

[2] sync does flush only a fraction of dirty buffers:
http://leo.kloburg.at/tmp/dirty-buffers/bad_slime-2.6.14-1.1653_FC4smp.txt