public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: "Arkadiusz Miśkiewicz" <arekm@maven.pl>
To: Michal Hocko <mhocko@suse.cz>
Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>,
	linux-mm@kvack.org, cl@linux.com, htejun@gmail.com,
	xfs@oss.sgi.com
Subject: Re: memory reclaim problems on fs usage
Date: Wed, 18 Nov 2015 23:36:18 +0100	[thread overview]
Message-ID: <201511182336.18231.arekm@maven.pl> (raw)
In-Reply-To: <20151116161518.GI14116@dhcp22.suse.cz>

On Monday 16 of November 2015, Michal Hocko wrote:
> On Sun 15-11-15 15:49:35, Arkadiusz Miśkiewicz wrote:
> > On Sunday 15 of November 2015, Tetsuo Handa wrote:
> > > Arkadiusz Miskiewicz wrote:
> > > > On Sunday 15 of November 2015, Tetsuo Handa wrote:
> > > > > I think that the vmstat statistics now have correct values.
> > > > > 
> > > > > > But are these patches solving the problem or just hiding it?
> > > > > 
> > > > > Excuse me but I can't judge.
> > > > > 
> > > > > If you are interested in monitoring how vmstat statistics are
> > > > > changing under stalled condition, you can try below patch.
> > > > 
> > > > Here is log with this and all previous patches applied:
> > > > http://ixion.pld-linux.org/~arekm/log-mm-5.txt.gz
> > > 
> > > Regarding "Node 0 Normal" (min:7104kB low:8880kB high:10656kB),
> > > all free: values look sane to me. I think that your problem was solved.
> > 
> > Great, thanks!
> > 
> > Will all (or part) of these patches
> > 
> > http://sprunge.us/GYBb
> 
> Migrate reserves are not a stable material I am afraid. "vmstat:
> explicitly schedule per-cpu work on the CPU we need it to run on"
> was not marked for stable either but I am not sure why it should make
> any difference for your load. I understand that testing this is really
> tedious but it would be better to know which of the patches actually
> made a difference.

Ok. In mean time I've tried 4.3.0 kernel + patches (the same as before + one 
more) on second server which runs even more rsnapshot processes and also uses 
xfs on md raid 6.

Patches:
http://sprunge.us/DfIQ (debug patch from Tetsuo)
http://sprunge.us/LQPF (backport of things from git + one from ml)

The problem is now with high order allocations probably:
http://ixion.pld-linux.org/~arekm/log-mm-2srv-1.txt.gz

System is doing very slow progress and for example depmod run took 2 hours
http://sprunge.us/HGbE
Sometimes I was able to ssh-in, dmesg took 10-15 minutes but sometimes it 
worked fast for short period.

Ideas?

ps. I also had one problem with low order allocation but only once and wasn't 
able to reproduce so far. I was running kernel with backport patches but no 
debug patch, so got only this in logs:
http://sprunge.us/WPXi

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2015-11-18 22:36 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-10 22:13 memory reclaim problems on fs usage Arkadiusz Miśkiewicz
2015-11-11 15:58 ` Tetsuo Handa
2015-11-11 16:19   ` Arkadiusz Miśkiewicz
2015-11-11 22:19     ` Tetsuo Handa
2015-11-12  6:06       ` Arkadiusz Miśkiewicz
2015-11-12 14:12         ` Tetsuo Handa
2015-11-12 20:06           ` Dave Chinner
2015-11-13 12:19             ` Tetsuo Handa
2015-11-12 21:28           ` Arkadiusz Miśkiewicz
2015-11-14 20:40             ` Arkadiusz Miśkiewicz
2015-11-15  2:35               ` Tetsuo Handa
2015-11-15 11:29                 ` Arkadiusz Miśkiewicz
2015-11-15 14:13                   ` Tetsuo Handa
2015-11-15 14:49                     ` Arkadiusz Miśkiewicz
2015-11-16 16:15                       ` Michal Hocko
2015-11-18 22:36                         ` Arkadiusz Miśkiewicz [this message]
2015-11-19 15:59                           ` Tetsuo Handa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=201511182336.18231.arekm@maven.pl \
    --to=arekm@maven.pl \
    --cc=cl@linux.com \
    --cc=htejun@gmail.com \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.cz \
    --cc=penguin-kernel@i-love.sakura.ne.jp \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox