Re: [PATCH] mm, oom: report compaction/migration stats for higher order requests

["Arkadiusz Miśkiewicz" <arekm@maven.pl>]

On Tuesday 16 of August 2016, Michal Hocko wrote:
> On Tue 16-08-16 13:18:25, Arkadiusz Miskiewicz wrote:
> > On Monday 15 of August 2016, Michal Hocko wrote:
> > > [Fixing up linux-mm]
> > > 
> > > Ups I had a c&p error in the previous patch. Here is an updated patch.
> > 
> > Going to apply this patch now and report again. I mean time what I have
> > is a
> > 
> >  while (true); do echo "XX date"; date; echo "XX SLAB"; cat
> >  /proc/slabinfo ;
> > 
> > echo "XX VMSTAT"; cat /proc/vmstat ; echo "XX free"; free; echo "XX
> > DMESG"; dmesg -T | tail -n 50; /bin/sleep 60;done 2>&1 | tee log
> > 
> > loop gathering some data while few OOM conditions happened.
> > 
> > I was doing "rm -rf copyX; cp -al original copyX" 10x in parallel.
> > 
> > https://ixion.pld-linux.org/~arekm/p2/ext4/log-20160816.txt
> 
> David was right when assuming it would be the ext4 inode cache which
> consumes the large portion of the memory. /proc/slabinfo shows
> ext4_inode_cache consuming between 2.5 to 4.6G of memory.
> 
> 			first value	last-first
> pgmigrate_success       1861785 	2157917
> pgmigrate_fail  	335344  	1400384
> compact_isolated        4106390 	5777027
> compact_migrate_scanned 113962774       446290647
> compact_daemon_wake     17039   	43981
> compact_fail    	645     	1039
> compact_free_scanned    381701557       793430119
> compact_success 	217     	307
> compact_stall   	862     	1346
> 
> which means that we have invoked compaction 1346 times and failed in
> 77% of cases. It is interesting to see that the migration wasn't all
> that unsuccessful. We managed to migrate 1.5x more pages than failed. It
> smells like the compaction just backs off.

With "[PATCH] mm, oom: report compaction/migration stats for higher order 
requests" patch:
https://ixion.pld-linux.org/~arekm/p2/ext4/log-20160817.txt

Didn't count much - all counters are 0
compaction_stall:0 compaction_fail:0 compact_migrate_scanned:0 
compact_free_scanned:0 compact_isolated:0 pgmigrate_success:0 pgmigrate_fail:0

two processes were killed by OOM (rm and cp), the rest of rm/cp didn't finish 
and I'm interrupting it to try that next patch:

> Could you try to test with
> patch from
> http://lkml.kernel.org/r/20160816031222.GC16913@js1304-P5Q-DELUXE please?
> Ideally on top of linux-next. You can add both the compaction counters
> patch in the oom report and high order atomic reserves patch on top.

Uhm, was going to use it on top of 4.7.[01] first.

> Thanks

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>