All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Ming Lei <ming.lei@canonical.com>
Cc: linux-kernel@vger.kernel.org,
	Alan Stern <stern@rowland.harvard.edu>,
	Oliver Neukum <oneukum@suse.de>, Minchan Kim <minchan@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rjw@sisk.pl>, Jens Axboe <axboe@kernel.dk>,
	"David S. Miller" <davem@davemloft.net>,
	netdev@vger.kernel.org, linux-usb@vger.kernel.org,
	linux-pm@vger.kernel.org, linux-mm@kvack.org,
	Jiri Kosina <jiri.kosina@suse.com>, Mel Gorman <mel@csn.ul.ie>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Michal Hocko <mhocko@suse.cz>, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH v4 1/6] mm: teach mm by current context info to not do I/O during memory allocation
Date: Tue, 6 Nov 2012 19:48:59 -0800	[thread overview]
Message-ID: <20121106194859.8eec3043.akpm@linux-foundation.org> (raw)
In-Reply-To: <CACVXFVNs2JtEYQ3Y2rA8L89sAaMJ7TO-PxG3h4w+ihcZrBLtpg@mail.gmail.com>

On Wed, 7 Nov 2012 11:11:24 +0800 Ming Lei <ming.lei@canonical.com> wrote:

> On Wed, Nov 7, 2012 at 7:23 AM, Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > It's unclear from the description why we're also clearing __GFP_FS in
> > this situation.
> >
> > If we can avoid doing this then there will be a very small gain: there
> > are some situations in which a filesystem can clean pagecache without
> > performing I/O.
> 
> Firstly,  the patch follows the policy in the system suspend/resume situation,
> in which the __GFP_FS is cleared, and basically the problem is very similar
> with that in system PM path.

I suspect that code is wrong.  Or at least, suboptimal.

> Secondly, inside shrink_page_list(), pageout() may be triggered on dirty anon
> page if __GFP_FS is set.

pageout() should be called if GFP_FS is set or if GFP_IO is set and the
IO is against swap.

And that's what we want to happen: we want to enter the fs to try to
turn dirty pagecache into clean pagecache without doing IO.  If we in
fact enter the device drivers when GFP_IO was not set then that's a bug
which we should fix.

> IMO, if performing I/O can be completely avoided when __GFP_FS is set, the
> flag can be kept, otherwise it is better to clear it in the situation.

yup.

> >
> > Also, you can probably put the unlikely() inside memalloc_noio() and
> > avoid repeating it at all the callsites.
> >
> > And it might be neater to do:
> >
> > /*
> >  * Nice comment goes here
> >  */
> > static inline gfp_t memalloc_noio_flags(gfp_t flags)
> > {
> >         if (unlikely(current->flags & PF_MEMALLOC_NOIO))
> >                 flags &= ~GFP_IOFS;
> >         return flags;
> > }
> 
> But without the check in callsites, some local variables will be write
> two times,
> so it is better to not do it.

I don't see why - we just modify the incoming gfp_t at the start of the
function, then use it.

It gets a bit tricky with those struct initialisations.  Things like

	struct foo bar {
		.a = a1,
		.b = b1,
	};

should not be turned into

	struct foo bar {
		.a = a1,
	};
	
	bar.b = b1;

and we don't want to do

	struct foo bar { };

	bar.a = a1;
	bar.b = b1;

either, because these are indeed a double-write.  But we can do

	struct foo bar {
		.flags = (flags = memalloc_noio_flags(flags)),
		.b = b1,
	};

which is a bit arcane but not toooo bad.  Have a think about it...


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Ming Lei <ming.lei@canonical.com>
Cc: linux-kernel@vger.kernel.org,
	Alan Stern <stern@rowland.harvard.edu>,
	Oliver Neukum <oneukum@suse.de>, Minchan Kim <minchan@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rjw@sisk.pl>, Jens Axboe <axboe@kernel.dk>,
	"David S. Miller" <davem@davemloft.net>,
	netdev@vger.kernel.org, linux-usb@vger.kernel.org,
	linux-pm@vger.kernel.org, linux-mm@kvack.org,
	Jiri Kosina <jiri.kosina@suse.com>, Mel Gorman <mel@csn.ul.ie>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Michal Hocko <mhocko@suse.cz>, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH v4 1/6] mm: teach mm by current context info to not do I/O during memory allocation
Date: Tue, 6 Nov 2012 19:48:59 -0800	[thread overview]
Message-ID: <20121106194859.8eec3043.akpm@linux-foundation.org> (raw)
In-Reply-To: <CACVXFVNs2JtEYQ3Y2rA8L89sAaMJ7TO-PxG3h4w+ihcZrBLtpg@mail.gmail.com>

On Wed, 7 Nov 2012 11:11:24 +0800 Ming Lei <ming.lei@canonical.com> wrote:

> On Wed, Nov 7, 2012 at 7:23 AM, Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > It's unclear from the description why we're also clearing __GFP_FS in
> > this situation.
> >
> > If we can avoid doing this then there will be a very small gain: there
> > are some situations in which a filesystem can clean pagecache without
> > performing I/O.
> 
> Firstly,  the patch follows the policy in the system suspend/resume situation,
> in which the __GFP_FS is cleared, and basically the problem is very similar
> with that in system PM path.

I suspect that code is wrong.  Or at least, suboptimal.

> Secondly, inside shrink_page_list(), pageout() may be triggered on dirty anon
> page if __GFP_FS is set.

pageout() should be called if GFP_FS is set or if GFP_IO is set and the
IO is against swap.

And that's what we want to happen: we want to enter the fs to try to
turn dirty pagecache into clean pagecache without doing IO.  If we in
fact enter the device drivers when GFP_IO was not set then that's a bug
which we should fix.

> IMO, if performing I/O can be completely avoided when __GFP_FS is set, the
> flag can be kept, otherwise it is better to clear it in the situation.

yup.

> >
> > Also, you can probably put the unlikely() inside memalloc_noio() and
> > avoid repeating it at all the callsites.
> >
> > And it might be neater to do:
> >
> > /*
> >  * Nice comment goes here
> >  */
> > static inline gfp_t memalloc_noio_flags(gfp_t flags)
> > {
> >         if (unlikely(current->flags & PF_MEMALLOC_NOIO))
> >                 flags &= ~GFP_IOFS;
> >         return flags;
> > }
> 
> But without the check in callsites, some local variables will be write
> two times,
> so it is better to not do it.

I don't see why - we just modify the incoming gfp_t at the start of the
function, then use it.

It gets a bit tricky with those struct initialisations.  Things like

	struct foo bar {
		.a = a1,
		.b = b1,
	};

should not be turned into

	struct foo bar {
		.a = a1,
	};
	
	bar.b = b1;

and we don't want to do

	struct foo bar { };

	bar.a = a1;
	bar.b = b1;

either, because these are indeed a double-write.  But we can do

	struct foo bar {
		.flags = (flags = memalloc_noio_flags(flags)),
		.b = b1,
	};

which is a bit arcane but not toooo bad.  Have a think about it...



  reply	other threads:[~2012-11-07  3:48 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-11-03  8:35 [PATCH v4 0/6] solve deadlock caused by memory allocation with I/O Ming Lei
2012-11-03  8:35 ` Ming Lei
2012-11-03  8:35 ` [PATCH v4 1/6] mm: teach mm by current context info to not do I/O during memory allocation Ming Lei
2012-11-03  8:35   ` Ming Lei
2012-11-06 23:23   ` Andrew Morton
2012-11-06 23:23     ` Andrew Morton
2012-11-07  3:11     ` Ming Lei
2012-11-07  3:11       ` Ming Lei
2012-11-07  3:48       ` Andrew Morton [this message]
2012-11-07  3:48         ` Andrew Morton
2012-11-07  4:35         ` Ming Lei
2012-11-07  4:35           ` Ming Lei
2012-11-03  8:35 ` [PATCH v4 2/6] PM / Runtime: introduce pm_runtime_set_memalloc_noio() Ming Lei
2012-11-03  8:35   ` Ming Lei
     [not found]   ` <1351931714-11689-3-git-send-email-ming.lei-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
2012-11-06 23:24     ` Andrew Morton
2012-11-06 23:24       ` Andrew Morton
2012-11-06 23:24       ` Andrew Morton
2012-11-07  3:32       ` Ming Lei
2012-11-07  3:32         ` Ming Lei
2012-11-03  8:35 ` [PATCH v4 3/6] block/genhd.c: apply pm_runtime_set_memalloc_noio on block devices Ming Lei
2012-11-03  8:35   ` Ming Lei
2012-11-06 23:24   ` Andrew Morton
2012-11-06 23:24     ` Andrew Morton
2012-11-03  8:35 ` [PATCH v4 4/6] net/core: apply pm_runtime_set_memalloc_noio on network devices Ming Lei
2012-11-03  8:35   ` Ming Lei
2012-11-03  8:35 ` [PATCH v4 5/6] PM / Runtime: force memory allocation with no I/O during Runtime PM callbcack Ming Lei
2012-11-03  8:35   ` Ming Lei
2012-11-03  8:35 ` [PATCH v4 6/6] USB: forbid memory allocation with I/O during bus reset Ming Lei
2012-11-03  8:35   ` Ming Lei
2012-11-06 23:23 ` [PATCH v4 0/6] solve deadlock caused by memory allocation with I/O Andrew Morton
2012-11-06 23:23   ` Andrew Morton
2012-11-06 23:23   ` Andrew Morton
2012-11-07  3:37   ` Ming Lei
2012-11-07  3:37     ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20121106194859.8eec3043.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=davem@davemloft.net \
    --cc=gregkh@linuxfoundation.org \
    --cc=jiri.kosina@suse.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=mel@csn.ul.ie \
    --cc=mhocko@suse.cz \
    --cc=minchan@kernel.org \
    --cc=ming.lei@canonical.com \
    --cc=mingo@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=oneukum@suse.de \
    --cc=peterz@infradead.org \
    --cc=rjw@sisk.pl \
    --cc=stern@rowland.harvard.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.