Re: zone_reclaimable() leads to livelock in __alloc_pages_slowpath()

Michal Hocko
2016-05-24T07:16:24Z

On Mon 23-05-16 17:14:19, Oleg Nesterov wrote:
> On 05/23, Michal Hocko wrote:
[...]
> > Could you add some tracing and see what are the numbers
> > above?
>
> with the patch below I can press Ctrl-C when it hangs, this breaks the
> endless loop and the output looks like
>
> vmscan: ZONE=ffffffff8189f180 0 scanned=0 pages=6
> vmscan: ZONE=ffffffff8189eb00 0 scanned=1 pages=0
> ...
> vmscan: ZONE=ffffffff8189eb00 0 scanned=2 pages=1
> vmscan: ZONE=ffffffff8189f180 0 scanned=4 pages=6
> ...
> vmscan: ZONE=ffffffff8189f180 0 scanned=4 pages=6
> vmscan: ZONE=ffffffff8189f180 0 scanned=4 pages=6
>
> the numbers are always small.

Small but scanned is not 0 and constant which means it either gets reset
repeatedly (something gets freed) or we have stopped scanning. Which
pattern can you see? I assume that the swap space is full at the time
(could you add get_nr_swap_pages() to the output). Also zone->name would
be better than the pointer.

I am trying to reproduce but your test case always hits the oom killer:

This is in a qemu x86_64 virtual machine:
# free
total used free shared buffers cached
Mem: 490212 96788 393424 0 3196 9976
-/+ buffers/cache: 83616 406596
Swap: 138236 57740 80496

I have tried with much larger swap space but no change except for the
run time of the test which is expected.

# grep "^processor" /proc/cpuinfo | wc -l
1

[... Skipped several previous attempts ...]
[ 695.215235] vmscan: XXX: zone:DMA32 nr_pages_scanned:0 reclaimable:20
[ 695.215245] vmscan: XXX: zone:DMA32 nr_pages_scanned:0 reclaimable:20
[ 695.215255] vmscan: XXX: zone:DMA32 nr_pages_scanned:0 reclaimable:20
[ 695.215282] vmscan: XXX: zone:DMA32 nr_pages_scanned:1 reclaimable:27
[ 695.215303] vmscan: XXX: zone:DMA32 nr_pages_scanned:5 reclaimable:27
[ 695.215327] vmscan: XXX: zone:DMA32 nr_pages_scanned:18 reclaimable:27
[ 695.215351] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215362] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215373] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215382] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215392] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215402] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215412] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215422] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215431] vmscan: XXX: zone:DMA32 nr_pages_scanned:45 reclaimable:27
[ 695.215442] vmscan: XXX: zone:DMA32 nr_pages_scanned:46 reclaimable:27
[ 695.215462] vmscan: XXX: zone:DMA32 nr_pages_scanned:48 reclaimable:27
[ 695.215482] vmscan: XXX: zone:DMA32 nr_pages_scanned:53 reclaimable:27
[ 695.215504] vmscan: XXX: zone:DMA32 nr_pages_scanned:63 reclaimable:27
[ 695.215528] vmscan: XXX: zone:DMA32 nr_pages_scanned:90 reclaimable:27
[...]
[ 695.215620] vmscan: XXX: zone:DMA32 nr_pages_scanned:91 reclaimable:27
[ 695.215640] vmscan: XXX: zone:DMA32 nr_pages_scanned:94 reclaimable:27
[ 695.215659] vmscan: XXX: zone:DMA32 nr_pages_scanned:100 reclaimable:27
[ 695.215683] vmscan: XXX: zone:DMA32 nr_pages_scanned:113 reclaimable:27
[...]
[ 695.215786] vmscan: XXX: zone:DMA32 nr_pages_scanned:140 reclaimable:27
[ 695.215797] vmscan: XXX: zone:DMA32 nr_pages_scanned:141 reclaimable:27
[ 695.215816] vmscan: XXX: zone:DMA32 nr_pages_scanned:144 reclaimable:27
[ 695.215836] vmscan: XXX: zone:DMA32 nr_pages_scanned:150 reclaimable:27
[ 695.215906] test-oleg invoked oom-killer: gfp_mask=0x24201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), order=0, oom_score_adj=0
--
Michal Hocko
SUSE Labs