From: Mel Gorman <mgorman@suse.de>
To: Dave Chinner <david@fromorbit.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Jiri Slaby <jslaby@suse.cz>,
Valdis Kletnieks <Valdis.Kletnieks@vt.edu>,
Rik van Riel <riel@redhat.com>,
Zlatko Calusic <zcalusic@bitsync.net>,
Johannes Weiner <hannes@cmpxchg.org>,
dormando <dormando@rydia.net>, Michal Hocko <mhocko@suse.cz>,
Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/9] Reduce system disruption due to kswapd V4
Date: Wed, 22 May 2013 09:48:59 +0100 [thread overview]
Message-ID: <20130522083722.GU11497@suse.de> (raw)
In-Reply-To: <20130521231358.GV29466@dastard>
On Wed, May 22, 2013 at 09:13:58AM +1000, Dave Chinner wrote:
> On Mon, May 13, 2013 at 09:12:31AM +0100, Mel Gorman wrote:
> > This series does not fix all the current known problems with reclaim but
> > it addresses one important swapping bug when there is background IO.
>
> ....
> >
> > 3.10.0-rc1 3.10.0-rc1
> > vanilla lessdisrupt-v4
> > Page Ins 1234608 101892
> > Page Outs 12446272 11810468
> > Swap Ins 283406 0
> > Swap Outs 698469 27882
> > Direct pages scanned 0 136480
> > Kswapd pages scanned 6266537 5369364
> > Kswapd pages reclaimed 1088989 930832
> > Direct pages reclaimed 0 120901
> > Kswapd efficiency 17% 17%
> > Kswapd velocity 5398.371 4635.115
> > Direct efficiency 100% 88%
> > Direct velocity 0.000 117.817
> > Percentage direct scans 0% 2%
> > Page writes by reclaim 1655843 4009929
> > Page writes file 957374 3982047
>
> Lots more file pages are written by reclaim. Is this from kswapd
> or direct reclaim? If it's direct reclaim, what happens when you run
> on a filesystem that doesn't allow writeback from direct reclaim?
>
It's from kswapd. There is a check in shrink_page_list that prevents direct
reclaim writing pages out for exactly the reason that some filesystems
ignore it.
> Also, what does this do to IO patterns and allocation? This tends
> to indicate that the background flusher thread is not doing the
> writeback work fast enough when memory is low - can you comment on
> this at all, Mel?
>
There are two aspects to it. As processes are not longer being pushed
to swap but kswapd is still reclaiming a similar number of pages, it is
scanning through the file LRUs faster before flushers have a chance to
flush pages. kswapd starts writing pages if the zone gets marked "reclaim
dirty" which happens if enough dirty pages are encountered at the end of
the LRU that are !PageWriteback. If this flag is set too early then more
writes from kswapd context occur -- I'll look into it.
On a related note, I've found with Jan Kara that the PageWriteback check
does not work in all cases. Some filesystems will have buffer pages that
are PageDirty with all clean buffers or with buffers locked for IO that are
!PageWriteback which will also confuse when "reclaim dirty" gets set. The
patches are still being a work in progress.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
prev parent reply other threads:[~2013-05-22 8:49 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-13 8:12 [PATCH 0/9] Reduce system disruption due to kswapd V4 Mel Gorman
2013-05-13 8:12 ` [PATCH 1/9] mm: vmscan: Limit the number of pages kswapd reclaims at each priority Mel Gorman
2013-05-13 8:12 ` [PATCH 2/9] mm: vmscan: Obey proportional scanning requirements for kswapd Mel Gorman
2013-05-14 10:21 ` Michal Hocko
2013-05-17 3:41 ` Kamezawa Hiroyuki
2013-05-13 8:12 ` [PATCH 3/9] mm: vmscan: Flatten kswapd priority loop Mel Gorman
2013-05-14 10:38 ` Michal Hocko
2013-05-13 8:12 ` [PATCH 4/9] mm: vmscan: Decide whether to compact the pgdat based on reclaim progress Mel Gorman
2013-05-14 10:51 ` Michal Hocko
2013-05-13 8:12 ` [PATCH 5/9] mm: vmscan: Do not allow kswapd to scan at maximum priority Mel Gorman
2013-05-13 8:12 ` [PATCH 6/9] mm: vmscan: Have kswapd writeback pages based on dirty pages encountered, not priority Mel Gorman
2013-05-14 11:25 ` Michal Hocko
2013-05-13 8:12 ` [PATCH 7/9] mm: vmscan: Block kswapd if it is encountering pages under writeback Mel Gorman
2013-05-14 21:06 ` Rik van Riel
2013-05-15 21:39 ` Andrew Morton
2013-05-16 13:07 ` Mel Gorman
2013-05-13 8:12 ` [PATCH 8/9] mm: vmscan: Check if kswapd should writepage once per pgdat scan Mel Gorman
2013-05-13 8:12 ` [PATCH 9/9] mm: vmscan: Move logic from balance_pgdat() to kswapd_shrink_zone() Mel Gorman
2013-05-14 12:23 ` Michal Hocko
2013-05-14 21:06 ` Rik van Riel
2013-05-15 20:37 ` [PATCH 0/9] Reduce system disruption due to kswapd V4 Andrew Morton
2013-05-16 10:33 ` Mel Gorman
2013-05-16 13:54 ` Michal Hocko
2013-05-16 14:11 ` Mel Gorman
2013-05-18 21:15 ` Zlatko Calusic
2013-05-21 23:13 ` Dave Chinner
2013-05-22 8:48 ` Mel Gorman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130522083722.GU11497@suse.de \
--to=mgorman@suse.de \
--cc=Valdis.Kletnieks@vt.edu \
--cc=akpm@linux-foundation.org \
--cc=david@fromorbit.com \
--cc=dormando@rydia.net \
--cc=hannes@cmpxchg.org \
--cc=jslaby@suse.cz \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.cz \
--cc=riel@redhat.com \
--cc=zcalusic@bitsync.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).