* 2.6.xx: dirty pages never being sync'd to disk?
@ 2005-11-14 15:30 Mark Lord
2005-11-14 15:35 ` Arjan van de Ven
0 siblings, 1 reply; 9+ messages in thread
From: Mark Lord @ 2005-11-14 15:30 UTC (permalink / raw)
To: Linux Kernel
Okay, this one's been nagging me since I first began using 2.6.xx.
My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite
happy to leave hundreds of MB of dirty unsync'd pages laying around
more or less indefinitely. This worries me, because that's a lot of data
to lose should the kernel crash (which it has once quite recently)
or the battery die.
/proc/sys/vm/dirty_expire_centisecs = 3000 (30 seconds)
/proc/sys/vm/dirty_writeback_centisecs = 500 (5 seconds)
My understanding (please correct if wrong) is that this means
that any (file data) page which is dirtied, should get flushed
back to disk after 30 seconds or so.
That doesn't happen here. Hundreds of MB of dirty pages just
hang around indefinitely, until I manually type "sync",
at which point the hard drive gets very busy for 20 seconds or so.
What's going on?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 15:30 2.6.xx: dirty pages never being sync'd to disk? Mark Lord
@ 2005-11-14 15:35 ` Arjan van de Ven
2005-11-14 15:49 ` Mark Lord
0 siblings, 1 reply; 9+ messages in thread
From: Arjan van de Ven @ 2005-11-14 15:35 UTC (permalink / raw)
To: Mark Lord; +Cc: Linux Kernel
On Mon, 2005-11-14 at 10:30 -0500, Mark Lord wrote:
> Okay, this one's been nagging me since I first began using 2.6.xx.
>
> My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite
> happy to leave hundreds of MB of dirty unsync'd pages laying around
> more or less indefinitely. This worries me, because that's a lot of data
> to lose should the kernel crash (which it has once quite recently)
> or the battery die.
>
> /proc/sys/vm/dirty_expire_centisecs = 3000 (30 seconds)
> /proc/sys/vm/dirty_writeback_centisecs = 500 (5 seconds)
>
> My understanding (please correct if wrong) is that this means
> that any (file data) page which is dirtied, should get flushed
> back to disk after 30 seconds or so.
do you have laptop mode enabled? That changes the behavior bigtime in
this regard and makes the kernel behave quite different.
also if these are files written to by mmap, the kernel only really sees
those as dirty when the mapping gets taken down (eg the propagation from
the pagetable dirty bit to the per page dirty bit goes a bit lazy)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 15:35 ` Arjan van de Ven
@ 2005-11-14 15:49 ` Mark Lord
2005-11-14 15:54 ` Mark Lord
2005-11-14 16:56 ` Badari Pulavarty
0 siblings, 2 replies; 9+ messages in thread
From: Mark Lord @ 2005-11-14 15:49 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: Linux Kernel
Arjan van de Ven wrote:
> On Mon, 2005-11-14 at 10:30 -0500, Mark Lord wrote:
..
>>My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite
>>happy to leave hundreds of MB of dirty unsync'd pages laying around
..
>>/proc/sys/vm/dirty_expire_centisecs = 3000 (30 seconds)
>>/proc/sys/vm/dirty_writeback_centisecs = 500 (5 seconds)
..
> do you have laptop mode enabled? That changes the behavior bigtime in
> this regard and makes the kernel behave quite different.
No. Laptop-mode mostly just modifies the dirty_expire
and related settings, and I have them set as shown above.
But there's also this:
/proc/sys/vm/laptop_mode = 0
> also if these are files written to by mmap, the kernel only really sees
> those as dirty when the mapping gets taken down
They certainly show up in the counts in /proc/meminfo under "Dirty",
so I assumed that means the kernel knows they are dirty.
A simple test I do for this:
$ mkdir t
$ cp /usr/src/*.bz2 t (about 400-500MB worth of kernel tar files)
In another window, I do this:
$ while (sleep 1); do echo -n "`date`: "; grep Dirty /proc/meminfo; done
And then watch the count get large, but take virtually forever
to count back down to a "safe" value.
Typing "sync" causes all the Dirty pages to immediately be flushed to disk,
as expected.
Here's what the monitoring of /proc/meminfo shows,
on an otherwise mostly idle system after having done
the big file copies noted earlier:
Mon Nov 14 10:40:22 EST 2005: Dirty: 481284 kB
Mon Nov 14 10:40:23 EST 2005: Dirty: 479680 kB
Mon Nov 14 10:40:24 EST 2005: Dirty: 480380 kB
Mon Nov 14 10:40:25 EST 2005: Dirty: 480380 kB
Mon Nov 14 10:40:26 EST 2005: Dirty: 480380 kB
Mon Nov 14 10:40:27 EST 2005: Dirty: 480380 kB
Mon Nov 14 10:40:28 EST 2005: Dirty: 480384 kB
Mon Nov 14 10:40:29 EST 2005: Dirty: 480384 kB
Mon Nov 14 10:40:30 EST 2005: Dirty: 480384 kB
Mon Nov 14 10:40:31 EST 2005: Dirty: 480384 kB
Mon Nov 14 10:40:32 EST 2005: Dirty: 480384 kB
Mon Nov 14 10:40:33 EST 2005: Dirty: 480688 kB
Mon Nov 14 10:40:34 EST 2005: Dirty: 479972 kB
Mon Nov 14 10:40:35 EST 2005: Dirty: 479972 kB
Mon Nov 14 10:40:36 EST 2005: Dirty: 479972 kB
Mon Nov 14 10:40:37 EST 2005: Dirty: 480016 kB
Mon Nov 14 10:40:38 EST 2005: Dirty: 480016 kB
Mon Nov 14 10:40:39 EST 2005: Dirty: 480016 kB
Mon Nov 14 10:40:40 EST 2005: Dirty: 480020 kB
Mon Nov 14 10:40:41 EST 2005: Dirty: 480020 kB
Mon Nov 14 10:40:42 EST 2005: Dirty: 480028 kB
Mon Nov 14 10:40:43 EST 2005: Dirty: 480028 kB
Mon Nov 14 10:40:44 EST 2005: Dirty: 475868 kB
Mon Nov 14 10:40:45 EST 2005: Dirty: 475868 kB
Mon Nov 14 10:40:46 EST 2005: Dirty: 475868 kB
Mon Nov 14 10:40:47 EST 2005: Dirty: 475868 kB
Mon Nov 14 10:40:48 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:49 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:50 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:51 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:52 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:53 EST 2005: Dirty: 475880 kB
Mon Nov 14 10:40:54 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:40:55 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:40:57 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:40:58 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:40:59 EST 2005: Dirty: 455164 kB
Mon Nov 14 10:41:00 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:41:01 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:41:02 EST 2005: Dirty: 455160 kB
Mon Nov 14 10:41:03 EST 2005: Dirty: 455164 kB
Mon Nov 14 10:41:04 EST 2005: Dirty: 455164 kB
Mon Nov 14 10:41:05 EST 2005: Dirty: 455168 kB
Mon Nov 14 10:41:06 EST 2005: Dirty: 455168 kB
Mon Nov 14 10:41:07 EST 2005: Dirty: 455168 kB
Mon Nov 14 10:41:08 EST 2005: Dirty: 455188 kB
Mon Nov 14 10:41:09 EST 2005: Dirty: 455176 kB
Mon Nov 14 10:41:10 EST 2005: Dirty: 455176 kB
Mon Nov 14 10:41:11 EST 2005: Dirty: 455176 kB
Mon Nov 14 10:41:12 EST 2005: Dirty: 455176 kB
Mon Nov 14 10:41:13 EST 2005: Dirty: 455180 kB
Mon Nov 14 10:41:14 EST 2005: Dirty: 450972 kB
Mon Nov 14 10:41:15 EST 2005: Dirty: 450972 kB
Mon Nov 14 10:41:16 EST 2005: Dirty: 450972 kB
Mon Nov 14 10:41:17 EST 2005: Dirty: 450972 kB
Mon Nov 14 10:41:18 EST 2005: Dirty: 451016 kB
Mon Nov 14 10:41:19 EST 2005: Dirty: 430336 kB
Mon Nov 14 10:41:20 EST 2005: Dirty: 430336 kB
Mon Nov 14 10:41:21 EST 2005: Dirty: 430336 kB
Mon Nov 14 10:41:22 EST 2005: Dirty: 430336 kB
Mon Nov 14 10:41:23 EST 2005: Dirty: 430348 kB
Mon Nov 14 10:41:24 EST 2005: Dirty: 430348 kB
Mon Nov 14 10:41:25 EST 2005: Dirty: 430348 kB
Mon Nov 14 10:41:26 EST 2005: Dirty: 430348 kB
Mon Nov 14 10:41:27 EST 2005: Dirty: 430348 kB
Mon Nov 14 10:41:28 EST 2005: Dirty: 430356 kB
Mon Nov 14 10:41:29 EST 2005: Dirty: 430352 kB
Mon Nov 14 10:41:30 EST 2005: Dirty: 430352 kB
Mon Nov 14 10:41:31 EST 2005: Dirty: 430352 kB
Mon Nov 14 10:41:32 EST 2005: Dirty: 430352 kB
Mon Nov 14 10:41:33 EST 2005: Dirty: 430356 kB
Mon Nov 14 10:41:34 EST 2005: Dirty: 430356 kB
Mon Nov 14 10:41:35 EST 2005: Dirty: 430356 kB
Mon Nov 14 10:41:36 EST 2005: Dirty: 430356 kB
Mon Nov 14 10:41:37 EST 2005: Dirty: 430368 kB
Mon Nov 14 10:41:38 EST 2005: Dirty: 430364 kB
Mon Nov 14 10:41:39 EST 2005: Dirty: 430360 kB
Mon Nov 14 10:41:40 EST 2005: Dirty: 430364 kB
Mon Nov 14 10:41:41 EST 2005: Dirty: 430364 kB
Mon Nov 14 10:41:42 EST 2005: Dirty: 430368 kB
Mon Nov 14 10:41:43 EST 2005: Dirty: 430368 kB
Mon Nov 14 10:41:44 EST 2005: Dirty: 405552 kB
Mon Nov 14 10:41:45 EST 2005: Dirty: 405552 kB
Mon Nov 14 10:41:46 EST 2005: Dirty: 405552 kB
Mon Nov 14 10:41:47 EST 2005: Dirty: 405552 kB
Mon Nov 14 10:41:48 EST 2005: Dirty: 405556 kB
Mon Nov 14 10:41:49 EST 2005: Dirty: 405548 kB
Mon Nov 14 10:41:50 EST 2005: Dirty: 405548 kB
Mon Nov 14 10:41:51 EST 2005: Dirty: 405548 kB
Mon Nov 14 10:41:52 EST 2005: Dirty: 405548 kB
Mon Nov 14 10:41:53 EST 2005: Dirty: 405552 kB
Mon Nov 14 10:41:54 EST 2005: Dirty: 405492 kB
Mon Nov 14 10:41:55 EST 2005: Dirty: 405492 kB
Mon Nov 14 10:41:56 EST 2005: Dirty: 405492 kB
Mon Nov 14 10:41:57 EST 2005: Dirty: 405524 kB
Mon Nov 14 10:41:58 EST 2005: Dirty: 405528 kB
Mon Nov 14 10:41:59 EST 2005: Dirty: 405524 kB
Mon Nov 14 10:42:00 EST 2005: Dirty: 405524 kB
Mon Nov 14 10:42:01 EST 2005: Dirty: 405524 kB
Mon Nov 14 10:42:02 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:03 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:04 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:05 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:06 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:07 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:08 EST 2005: Dirty: 405536 kB
Mon Nov 14 10:42:10 EST 2005: Dirty: 405532 kB
Mon Nov 14 10:42:11 EST 2005: Dirty: 405532 kB
Mon Nov 14 10:42:12 EST 2005: Dirty: 405532 kB
Mon Nov 14 10:42:13 EST 2005: Dirty: 405532 kB
Mon Nov 14 10:42:14 EST 2005: Dirty: 405544 kB
Mon Nov 14 10:42:15 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:16 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:17 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:18 EST 2005: Dirty: 380680 kB
Mon Nov 14 10:42:19 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:20 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:21 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:22 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:23 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:24 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:25 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:26 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:27 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:28 EST 2005: Dirty: 380680 kB
Mon Nov 14 10:42:29 EST 2005: Dirty: 380668 kB
Mon Nov 14 10:42:30 EST 2005: Dirty: 380668 kB
Mon Nov 14 10:42:31 EST 2005: Dirty: 380668 kB
Mon Nov 14 10:42:32 EST 2005: Dirty: 380668 kB
Mon Nov 14 10:42:33 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:34 EST 2005: Dirty: 380628 kB
Mon Nov 14 10:42:35 EST 2005: Dirty: 380628 kB
Mon Nov 14 10:42:36 EST 2005: Dirty: 380628 kB
Mon Nov 14 10:42:37 EST 2005: Dirty: 380632 kB
Mon Nov 14 10:42:38 EST 2005: Dirty: 380672 kB
Mon Nov 14 10:42:39 EST 2005: Dirty: 380672 kB
Mon Nov 14 10:42:40 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:41 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:42 EST 2005: Dirty: 380676 kB
Mon Nov 14 10:42:43 EST 2005: Dirty: 380684 kB
Mon Nov 14 10:42:44 EST 2005: Dirty: 362476 kB
Mon Nov 14 10:42:45 EST 2005: Dirty: 362476 kB
Mon Nov 14 10:42:46 EST 2005: Dirty: 362476 kB
Mon Nov 14 10:42:47 EST 2005: Dirty: 362476 kB
Mon Nov 14 10:42:48 EST 2005: Dirty: 362476 kB
Mon Nov 14 10:42:49 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:50 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:51 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:52 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:53 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:54 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:55 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:56 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:57 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:42:58 EST 2005: Dirty: 358344 kB
Mon Nov 14 10:42:59 EST 2005: Dirty: 358344 kB
Mon Nov 14 10:43:00 EST 2005: Dirty: 358344 kB
Mon Nov 14 10:43:01 EST 2005: Dirty: 358344 kB
Mon Nov 14 10:43:02 EST 2005: Dirty: 358352 kB
Mon Nov 14 10:43:03 EST 2005: Dirty: 358352 kB
Mon Nov 14 10:43:04 EST 2005: Dirty: 358348 kB
Mon Nov 14 10:43:05 EST 2005: Dirty: 358348 kB
Mon Nov 14 10:43:06 EST 2005: Dirty: 358348 kB
Mon Nov 14 10:43:07 EST 2005: Dirty: 358348 kB
Mon Nov 14 10:43:08 EST 2005: Dirty: 358352 kB
Mon Nov 14 10:43:09 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:43:10 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:43:11 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:43:12 EST 2005: Dirty: 358340 kB
Mon Nov 14 10:43:13 EST 2005: Dirty: 358344 kB
Mon Nov 14 10:43:14 EST 2005: Dirty: 341716 kB
Mon Nov 14 10:43:15 EST 2005: Dirty: 341716 kB
Mon Nov 14 10:43:16 EST 2005: Dirty: 341716 kB
Mon Nov 14 10:43:17 EST 2005: Dirty: 341756 kB
Mon Nov 14 10:43:18 EST 2005: Dirty: 341756 kB
Mon Nov 14 10:43:19 EST 2005: Dirty: 341748 kB
Mon Nov 14 10:43:21 EST 2005: Dirty: 341748 kB
Mon Nov 14 10:43:22 EST 2005: Dirty: 341748 kB
Mon Nov 14 10:43:23 EST 2005: Dirty: 341752 kB
Mon Nov 14 10:43:24 EST 2005: Dirty: 341752 kB
Mon Nov 14 10:43:25 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:26 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:27 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:28 EST 2005: Dirty: 338276 kB
Mon Nov 14 10:43:29 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:30 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:31 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:32 EST 2005: Dirty: 338268 kB
Mon Nov 14 10:43:33 EST 2005: Dirty: 338272 kB
Mon Nov 14 10:43:34 EST 2005: Dirty: 338272 kB
Mon Nov 14 10:43:35 EST 2005: Dirty: 338272 kB
Mon Nov 14 10:43:36 EST 2005: Dirty: 338272 kB
Mon Nov 14 10:43:37 EST 2005: Dirty: 338276 kB
Mon Nov 14 10:43:38 EST 2005: Dirty: 338280 kB
Mon Nov 14 10:43:39 EST 2005: Dirty: 338276 kB
Mon Nov 14 10:43:40 EST 2005: Dirty: 338280 kB
Mon Nov 14 10:43:41 EST 2005: Dirty: 338280 kB
Mon Nov 14 10:43:42 EST 2005: Dirty: 338280 kB
Mon Nov 14 10:43:43 EST 2005: Dirty: 338288 kB
Mon Nov 14 10:43:44 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:45 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:46 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:47 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:48 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:49 EST 2005: Dirty: 321704 kB
Mon Nov 14 10:43:50 EST 2005: Dirty: 321704 kB
Mon Nov 14 10:43:51 EST 2005: Dirty: 321704 kB
Mon Nov 14 10:43:52 EST 2005: Dirty: 321704 kB
Mon Nov 14 10:43:53 EST 2005: Dirty: 321708 kB
Mon Nov 14 10:43:54 EST 2005: Dirty: 321656 kB
Mon Nov 14 10:43:55 EST 2005: Dirty: 321656 kB
Mon Nov 14 10:43:56 EST 2005: Dirty: 321656 kB
Mon Nov 14 10:43:57 EST 2005: Dirty: 321656 kB
Mon Nov 14 10:43:58 EST 2005: Dirty: 321688 kB
Mon Nov 14 10:43:59 EST 2005: Dirty: 321684 kB
Mon Nov 14 10:44:00 EST 2005: Dirty: 321684 kB
Mon Nov 14 10:44:01 EST 2005: Dirty: 321684 kB
Mon Nov 14 10:44:02 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:03 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:04 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:05 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:06 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:07 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:08 EST 2005: Dirty: 321696 kB
Mon Nov 14 10:44:09 EST 2005: Dirty: 321692 kB
Mon Nov 14 10:44:10 EST 2005: Dirty: 321692 kB
Mon Nov 14 10:44:11 EST 2005: Dirty: 321692 kB
Mon Nov 14 10:44:12 EST 2005: Dirty: 321692 kB
Mon Nov 14 10:44:13 EST 2005: Dirty: 321692 kB
Mon Nov 14 10:44:14 EST 2005: Dirty: 317604 kB
Mon Nov 14 10:44:15 EST 2005: Dirty: 317604 kB
Mon Nov 14 10:44:16 EST 2005: Dirty: 317608 kB
Mon Nov 14 10:44:17 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:18 EST 2005: Dirty: 317616 kB
Mon Nov 14 10:44:19 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:20 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:21 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:22 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:23 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:24 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:25 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:26 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:27 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:28 EST 2005: Dirty: 317616 kB
Mon Nov 14 10:44:29 EST 2005: Dirty: 317608 kB
Mon Nov 14 10:44:30 EST 2005: Dirty: 317608 kB
Mon Nov 14 10:44:32 EST 2005: Dirty: 317608 kB
Mon Nov 14 10:44:33 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:34 EST 2005: Dirty: 317612 kB
Mon Nov 14 10:44:35 EST 2005: Dirty: 317564 kB
Mon Nov 14 10:44:36 EST 2005: Dirty: 317564 kB
Mon Nov 14 10:44:37 EST 2005: Dirty: 317568 kB
Mon Nov 14 10:44:38 EST 2005: Dirty: 317608 kB
Mon Nov 14 10:44:39 EST 2005: Dirty: 317616 kB
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 15:49 ` Mark Lord
@ 2005-11-14 15:54 ` Mark Lord
2005-11-14 16:56 ` Badari Pulavarty
1 sibling, 0 replies; 9+ messages in thread
From: Mark Lord @ 2005-11-14 15:54 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: Linux Kernel
Here are all of the values from /proc/sys/vm/
0 block_dump
10 dirty_background_ratio
3000 dirty_expire_centisecs
40 dirty_ratio
500 dirty_writeback_centisecs
0 laptop_mode
0 legacy_va_layout
256 32 lowmem_reserve_ratio
65536 max_map_count
3831 min_free_kbytes
2 nr_pdflush_threads
0 overcommit_memory
50 overcommit_ratio
3 page-cluster
60 swappiness
300 swap_token_timeout
100 vfs_cache_pressure
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 15:49 ` Mark Lord
2005-11-14 15:54 ` Mark Lord
@ 2005-11-14 16:56 ` Badari Pulavarty
2005-11-14 17:15 ` Mark Lord
1 sibling, 1 reply; 9+ messages in thread
From: Badari Pulavarty @ 2005-11-14 16:56 UTC (permalink / raw)
To: Mark Lord; +Cc: Arjan van de Ven, Linux Kernel
On Mon, 2005-11-14 at 10:49 -0500, Mark Lord wrote:
> Arjan van de Ven wrote:
> > On Mon, 2005-11-14 at 10:30 -0500, Mark Lord wrote:
> ..
> >>My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite
> >>happy to leave hundreds of MB of dirty unsync'd pages laying around
> ..
> >>/proc/sys/vm/dirty_expire_centisecs = 3000 (30 seconds)
> >>/proc/sys/vm/dirty_writeback_centisecs = 500 (5 seconds)
> ..
> > do you have laptop mode enabled? That changes the behavior bigtime in
> > this regard and makes the kernel behave quite different.
>
> No. Laptop-mode mostly just modifies the dirty_expire
> and related settings, and I have them set as shown above.
> But there's also this:
>
> /proc/sys/vm/laptop_mode = 0
>
> > also if these are files written to by mmap, the kernel only really sees
> > those as dirty when the mapping gets taken down
>
> They certainly show up in the counts in /proc/meminfo under "Dirty",
> so I assumed that means the kernel knows they are dirty.
>
> A simple test I do for this:
>
> $ mkdir t
> $ cp /usr/src/*.bz2 t (about 400-500MB worth of kernel tar files)
>
> In another window, I do this:
>
> $ while (sleep 1); do echo -n "`date`: "; grep Dirty /proc/meminfo; done
>
> And then watch the count get large, but take virtually forever
> to count back down to a "safe" value.
>
> Typing "sync" causes all the Dirty pages to immediately be flushed to disk,
> as expected.
>
> Here's what the monitoring of /proc/meminfo shows,
> on an otherwise mostly idle system after having done
> the big file copies noted earlier:
>
> Mon Nov 14 10:40:22 EST 2005: Dirty: 481284 kB
> Mon Nov 14 10:40:23 EST 2005: Dirty: 479680 kB
Interesting. Since you have a very easy to reproduce case -
can you write a program to do posix_fadvise(POSIX_FADV_DONTNEED)
on those files in directory "t" and see what happens ?
Thanks,
Badari
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 16:56 ` Badari Pulavarty
@ 2005-11-14 17:15 ` Mark Lord
2005-11-14 17:38 ` Mark Lord
2005-11-14 17:44 ` Mark Lord
0 siblings, 2 replies; 9+ messages in thread
From: Mark Lord @ 2005-11-14 17:15 UTC (permalink / raw)
To: Badari Pulavarty; +Cc: Arjan van de Ven, Linux Kernel
Badari Pulavarty wrote:
>
> Interesting. Since you have a very easy to reproduce case -
> can you write a program to do posix_fadvise(POSIX_FADV_DONTNEED)
> on those files in directory "t" and see what happens ?
Sure. It appears to cause an immediate "sync" of the file data
to disk (lots of drive activity for a few seconds), and the Dirty
count from /proc/meminfo reduces correspondingly.
Oddly enough, the Dirty count didn't go all the way down, though.
Doing a "sync" or a second run of the program afterwards does
get the count down to zero immediately, without any significant I/O.
Strange.
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
int main (int argc, char *argv[])
{
while (--argc > 0) {
int fd = open(*++argv, O_RDWR);
if (fd == -1) {
perror(*argv);
} else {
int posix_fadvise(int, off_t, off_t, int);
const int POSIX_FADV_DONTNEED = 4;
int rc = posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
if (rc == -1)
perror(*argv);
close(fd);
}
}
return 0;
}
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 17:15 ` Mark Lord
@ 2005-11-14 17:38 ` Mark Lord
2005-11-14 17:44 ` Mark Lord
1 sibling, 0 replies; 9+ messages in thread
From: Mark Lord @ 2005-11-14 17:38 UTC (permalink / raw)
To: Mark Lord; +Cc: Badari Pulavarty, Arjan van de Ven, Linux Kernel
Ahh... there's the magic line I needed:
#define _GNU_SOURCE
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 17:15 ` Mark Lord
2005-11-14 17:38 ` Mark Lord
@ 2005-11-14 17:44 ` Mark Lord
2005-11-14 17:51 ` Mark Lord
1 sibling, 1 reply; 9+ messages in thread
From: Mark Lord @ 2005-11-14 17:44 UTC (permalink / raw)
To: Mark Lord; +Cc: Badari Pulavarty, Arjan van de Ven, Linux Kernel
Another intriguing observation:
If I do my file copy test from files that are already cached,
then the data usually gets committed to disk more or less
right away.
If the copy is done from uncached files, then the data usually
does not get committed to disk until I "sync" or run the
posix_fadvise test program on it.
Anyone else seeing this? 2.6.15-rc1, x86, 2GB
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6.xx: dirty pages never being sync'd to disk?
2005-11-14 17:44 ` Mark Lord
@ 2005-11-14 17:51 ` Mark Lord
0 siblings, 0 replies; 9+ messages in thread
From: Mark Lord @ 2005-11-14 17:51 UTC (permalink / raw)
To: Mark Lord; +Cc: Badari Pulavarty, Arjan van de Ven, Linux Kernel
Mark Lord wrote:
> Another intriguing observation:
>
> If I do my file copy test from files that are already cached,
> then the data usually gets committed to disk more or less
> right away.
And even in that case, /proc/meminfo still shows the memory
as "Dirty", until a "sync" is done. Why is that?
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2005-11-14 17:51 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-11-14 15:30 2.6.xx: dirty pages never being sync'd to disk? Mark Lord
2005-11-14 15:35 ` Arjan van de Ven
2005-11-14 15:49 ` Mark Lord
2005-11-14 15:54 ` Mark Lord
2005-11-14 16:56 ` Badari Pulavarty
2005-11-14 17:15 ` Mark Lord
2005-11-14 17:38 ` Mark Lord
2005-11-14 17:44 ` Mark Lord
2005-11-14 17:51 ` Mark Lord
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox