linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Trevor Cordes <trevor@tecnopolis.ca>
To: Mel Gorman <mgorman@techsingularity.net>
Cc: Michal Hocko <mhocko@kernel.org>,
	linux-kernel@vger.kernel.org,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@surriel.com>,
	Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Subject: Re: mm, vmscan: commit makes PAE kernel crash nightly (bisected)
Date: Wed, 18 Jan 2017 00:52:42 -0600	[thread overview]
Message-ID: <20170118005242.23379a74@pog.tecnopolis.ca> (raw)
In-Reply-To: <20170116110934.7zopy3ecg2lfadkd@techsingularity.net>

On 2017-01-16 Mel Gorman wrote:
> > > You can easily check whether this is memcg related by trying to
> > > run the same workload with cgroup_disable=memory kernel command
> > > line parameter. This will put all the memcg specifics out of the
> > > way.  
> > 
> > I will try booting now into cgroup_disable=memory to see if that
> > helps at all.  I'll reply back in 48 hours, or when it oom's,
> > whichever comes first.
> >   
> 
> Thanks.

It has successfully survived 70 hours and 2 3am cycles (when it
normally oom's) with your first patch *and* cgroup_disable=memory
grafted on Fedora's 4.8.13.  Since it has never survived 2 3am cycles,
I strongly suspect the cgroup_disable=memory mitigates my bug.

> > Also, should I bother trying the latest git HEAD to see if that
> > solves anything?  Thanks!  
> 
> That's worth trying. If that also fails then could you try the
> following hack to encourage direct reclaim to reclaim slab when
> buffers are over the limit please?
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 532a2a750952..46aac487b89a 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2684,6 +2684,7 @@ static void shrink_zones(struct zonelist
> *zonelist, struct scan_control *sc) continue;
>  
>  			if (sc->priority != DEF_PRIORITY &&
> +			    !buffer_heads_over_limit &&
>  			    !pgdat_reclaimable(zone->zone_pgdat))
>  				continue;	/* Let kswapd poll
> it */ 

What's the next best step?  HEAD?  HEAD + the above patch?  A new
patch?  I'll start a HEAD compile until I hear more.  I assume I should
test without cgroup_disable=memory as that's just a kludge/workaround,
right?

Also, is there a way to spot the slab pressure you are talking about
before oom's occur?  slabinfo?  I suppose I'd be able to see some
counter slowly getting too high or low?  Thanks!

  parent reply	other threads:[~2017-01-18  6:57 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-11 10:32 mm, vmscan: commit makes PAE kernel crash nightly (bisected) Trevor Cordes
2017-01-11 12:11 ` Mel Gorman
2017-01-11 12:14   ` Mel Gorman
2017-01-11 22:52     ` Trevor Cordes
2017-01-12  9:36       ` Michal Hocko
2017-01-15  6:27         ` Trevor Cordes
2017-01-16 11:09           ` Mel Gorman
2017-01-17 13:52             ` Michal Hocko
2017-01-17 14:21               ` Mel Gorman
2017-01-17 14:54                 ` Michal Hocko
2017-01-18  7:25                   ` Trevor Cordes
2017-01-18 17:48                   ` Mel Gorman
2017-01-18 18:07                   ` Mel Gorman
2017-01-19  9:48                   ` Trevor Cordes
2017-01-19 11:37                     ` Michal Hocko
2017-01-20  6:35                       ` Trevor Cordes
2017-01-20 11:02                         ` Mel Gorman
2017-01-20 15:55                           ` Mel Gorman
2017-01-23  0:45                             ` Trevor Cordes
2017-01-23 10:48                               ` Mel Gorman
2017-01-23 11:04                                 ` Mel Gorman
2017-01-25  9:46                                   ` Michal Hocko
2017-01-24 12:59                                 ` Michal Hocko
2017-01-25 10:02                                 ` Trevor Cordes
2017-01-25 12:04                                   ` Michal Hocko
2017-01-29 22:50                                     ` Trevor Cordes
2017-01-30  7:51                                       ` Michal Hocko
2017-02-01  9:29                                         ` Trevor Cordes
2017-02-01 10:14                                           ` Michal Hocko
2017-02-04  0:36                                             ` Trevor Cordes
2017-02-04 20:05                                               ` Rik van Riel
2017-02-05 10:03                                               ` Michal Hocko
2017-02-05 22:53                                                 ` Trevor Cordes
2017-01-30  9:10                                       ` Mel Gorman
2017-01-24 12:54                               ` Michal Hocko
2017-01-26 23:18                                 ` Trevor Cordes
2017-01-27  7:36                                   ` Michal Hocko
2017-01-24 12:51                         ` Michal Hocko
2017-01-18  6:52             ` Trevor Cordes [this message]
2017-01-17 13:45           ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170118005242.23379a74@pog.tecnopolis.ca \
    --to=trevor@tecnopolis.ca \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@kernel.org \
    --cc=minchan@kernel.org \
    --cc=riel@surriel.com \
    --cc=srikar@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).