Re: vmscan.c heuristic adjustment for smaller systems

Marc Singer
2004-04-18T06:15:37Z

On Sat, Apr 17, 2004 at 10:52:43PM -0700, Andrew Morton wrote:
> Well some more descriptions of what the system does after that copy-to-nfs
> would help. Does it _ever_ come good, or is a reboot needed, etc?

As far as I can tell, it never gets good. I run the same ls command
over and over. Ten times. Always the same behavior. Sometimes it
gets even slower. I can set swappiness to zero and it responds
normally, immediately.

> What does `vmstat 1' say during the copy, and during the ls?

I thought I sent a message about this. I've found that the problem
*only* occurs when there is exactly one process running. If I open a
second console (via telnet) then the slow-down behavior disappears.
If I logout of the console session and run the tests from the telnet
session then I *do* see the problem again.

> /proc/vmstats before and after the ls.

This one I can do.

BEFORE

nr_dirty 0
nr_writeback 0
nr_unstable 0
nr_page_table_pages 49
nr_mapped 163
nr_slab 255
pgpgin 4
pgpgout 0
pswpin 0
pswpout 0
pgalloc_high 0
pgalloc_normal 0
pgalloc_dma 78568
pgfree 79274
pgactivate 8428
pgdeactivate 8324
pgfault 17112
pgmajfault 1348
pgrefill_high 0
pgrefill_normal 0
pgrefill_dma 543296
pgsteal_high 0
pgsteal_normal 0
pgsteal_dma 60834
pgscan_kswapd_high 0
pgscan_kswapd_normal 0
pgscan_kswapd_dma 189700
pgscan_direct_high 0
pgscan_direct_normal 0
pgscan_direct_dma 31746
pginodesteal 0
slabs_scanned 1586
kswapd_steal 30907
kswapd_inodesteal 0
pageoutrun 77994
allocstall 82
pgrotated 0

ls -l /proc runs slowly.

AFTER

nr_dirty 0
nr_writeback 0
nr_unstable 0
nr_page_table_pages 49
nr_mapped 164
nr_slab 225
pgpgin 4
pgpgout 0
pswpin 0
pswpout 0
pgalloc_high 0
pgalloc_normal 0
pgalloc_dma 85378
pgfree 86106
pgactivate 11759
pgdeactivate 11650
pgfault 21293
pgmajfault 2316
pgrefill_high 0
pgrefill_normal 0
pgrefill_dma 616785
pgsteal_high 0
pgsteal_normal 0
pgsteal_dma 67511
pgscan_kswapd_high 0
pgscan_kswapd_normal 0
pgscan_kswapd_dma 200241
pgscan_direct_high 0
pgscan_direct_normal 0
pgscan_direct_dma 31746
pginodesteal 0
slabs_scanned 1586
kswapd_steal 37584
kswapd_inodesteal 0
pageoutrun 78405
allocstall 82
pgrotated 0

ls -l still slow.

Anything interesting?

> Try doing the copy, then when it has finished do the old
> memset(malloc(24M)) and monitor the `vmstat 1' output while it runs,
> capture /proc/meminfo before and after.

Here's the meminfo version of the same test above.

BEFORE

MemTotal: 30256 kB
MemFree: 3312 kB
Buffers: 0 kB
Cached: 24512 kB
SwapCached: 0 kB
Active: 732 kB
Inactive: 24084 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 30256 kB
LowFree: 3312 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
Mapped: 700 kB
Slab: 1024 kB
Committed_AS: 476 kB
PageTables: 196 kB
VmallocTotal: 434176 kB
VmallocUsed: 65924 kB
VmallocChunk: 368252 kB

ls -l runs slowly

AFTER

MemTotal: 30256 kB
MemFree: 3420 kB
Buffers: 0 kB
Cached: 24448 kB
SwapCached: 0 kB
Active: 772 kB
Inactive: 23984 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 30256 kB
LowFree: 3420 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
Mapped: 692 kB
Slab: 964 kB
Committed_AS: 476 kB
PageTables: 196 kB
VmallocTotal: 434176 kB
VmallocUsed: 65924 kB
VmallocChunk: 368252 kB

ls -l still runs slowly.

The copy w/vmstat involves two processes. It doesn't exhibit the
problems.

> None of the problems you report are present on x86 as far as I can tell,
> so...

I don't expect you would. Are you confident that this doesn't happen
on a 386 NFS root mounted system in single-user mode? I don't have an
IA32 system where I can test this scenario.