All of lore.kernel.org
 help / color / mirror / Atom feed
From: Benjamin LaHaise <bcrl@kvack.org>
To: Zach Brown <zab@redhat.com>
Cc: Tang Chen <tangchen@cn.fujitsu.com>, Mel Gorman <mgorman@suse.de>,
	Minchan Kim <minchan@kernel.org>,
	Lin Feng <linfeng@cn.fujitsu.com>,
	akpm@linux-foundation.org, viro@zeniv.linux.org.uk,
	khlebnikov@openvz.org, walken@google.com,
	kamezawa.hiroyu@jp.fujitsu.com, riel@redhat.com,
	rientjes@google.com, isimatu.yasuaki@jp.fujitsu.com,
	wency@cn.fujitsu.com, laijs@cn.fujitsu.com, jiang.liu@huawei.com,
	jmoyer@redhat.com, linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
	Marek Szyprowski <m.szyprowski@samsung.com>
Subject: Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())
Date: Fri, 17 May 2013 14:30:03 -0400	[thread overview]
Message-ID: <20130517183003.GL1008@kvack.org> (raw)
In-Reply-To: <20130517181708.GG318@lenny.home.zabbo.net>

On Fri, May 17, 2013 at 11:17:08AM -0700, Zach Brown wrote:
> > I ended up working on this a bit today, and managed to cobble together 
> > something that somewhat works -- please see the patch below.
> 
> Just some quick observations:
> 
> > +	ctx->ctx_file = anon_inode_getfile("[aio]", &aio_ctx_fops, ctx, O_RDWR);
> > +	if (IS_ERR(ctx->ctx_file)) {
> > +		ctx->ctx_file = NULL;
> > +		return -EAGAIN;
> > +	}
> 
> It's too bad that aio contexts will now be accounted against the filp
> limits (get_empty_filp -> files_stat.max_files, etc). 

Yeah, that is a downside of this approach.  It would be possible to to 
do it with only an inode/address_space, but that would mean bypassing 
do_mmap(), which is not worth considering.  If it is really an issue, we 
could add a flag to bypass that limit since aio has its own.  
anon_inode_getfile() as it stands is a major problem.

> > +	for (i=0; i<nr_pages; i++) {
> > +		struct page *page;
> > +		void *ptr;
> > +		page = find_or_create_page(ctx->ctx_file->f_inode->i_mapping,
> > +					   i, GFP_KERNEL);
> > +		if (!page) {
> > +			break;
> > +		}
> > +		ptr = kmap(page);
> > +		clear_page(ptr);
> > +		kunmap(page);
> > +		SetPageUptodate(page);
> > +		SetPageDirty(page);
> > +		unlock_page(page);
> > +	}
> 
> If they're GFP_KERNEL then you don't need to kmap them.  But we probably
> want to allocate with GFP_HIGHUSER and then use clear_user_highpage() to
> zero them?

Adding __GFP_ZERO would fix that too. The next respin will include that 
change.  I also have to properly handle the mremap() case as well.

		-ben
-- 
"Thought is the essence of where you are now."

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to majordomo@kvack.org.  For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>

WARNING: multiple messages have this Message-ID (diff)
From: Benjamin LaHaise <bcrl@kvack.org>
To: Zach Brown <zab@redhat.com>
Cc: Tang Chen <tangchen@cn.fujitsu.com>, Mel Gorman <mgorman@suse.de>,
	Minchan Kim <minchan@kernel.org>,
	Lin Feng <linfeng@cn.fujitsu.com>,
	akpm@linux-foundation.org, viro@zeniv.linux.org.uk,
	khlebnikov@openvz.org, walken@google.com,
	kamezawa.hiroyu@jp.fujitsu.com, riel@redhat.com,
	rientjes@google.com, isimatu.yasuaki@jp.fujitsu.com,
	wency@cn.fujitsu.com, laijs@cn.fujitsu.com, jiang.liu@huawei.com,
	jmoyer@redhat.com, linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
	Marek Szyprowski <m.szyprowski@samsung.com>
Subject: Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())
Date: Fri, 17 May 2013 14:30:03 -0400	[thread overview]
Message-ID: <20130517183003.GL1008@kvack.org> (raw)
In-Reply-To: <20130517181708.GG318@lenny.home.zabbo.net>

On Fri, May 17, 2013 at 11:17:08AM -0700, Zach Brown wrote:
> > I ended up working on this a bit today, and managed to cobble together 
> > something that somewhat works -- please see the patch below.
> 
> Just some quick observations:
> 
> > +	ctx->ctx_file = anon_inode_getfile("[aio]", &aio_ctx_fops, ctx, O_RDWR);
> > +	if (IS_ERR(ctx->ctx_file)) {
> > +		ctx->ctx_file = NULL;
> > +		return -EAGAIN;
> > +	}
> 
> It's too bad that aio contexts will now be accounted against the filp
> limits (get_empty_filp -> files_stat.max_files, etc). 

Yeah, that is a downside of this approach.  It would be possible to to 
do it with only an inode/address_space, but that would mean bypassing 
do_mmap(), which is not worth considering.  If it is really an issue, we 
could add a flag to bypass that limit since aio has its own.  
anon_inode_getfile() as it stands is a major problem.

> > +	for (i=0; i<nr_pages; i++) {
> > +		struct page *page;
> > +		void *ptr;
> > +		page = find_or_create_page(ctx->ctx_file->f_inode->i_mapping,
> > +					   i, GFP_KERNEL);
> > +		if (!page) {
> > +			break;
> > +		}
> > +		ptr = kmap(page);
> > +		clear_page(ptr);
> > +		kunmap(page);
> > +		SetPageUptodate(page);
> > +		SetPageDirty(page);
> > +		unlock_page(page);
> > +	}
> 
> If they're GFP_KERNEL then you don't need to kmap them.  But we probably
> want to allocate with GFP_HIGHUSER and then use clear_user_highpage() to
> zero them?

Adding __GFP_ZERO would fix that too. The next respin will include that 
change.  I also have to properly handle the mremap() case as well.

		-ben
-- 
"Thought is the essence of where you are now."

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Benjamin LaHaise <bcrl@kvack.org>
To: Zach Brown <zab@redhat.com>
Cc: Tang Chen <tangchen@cn.fujitsu.com>, Mel Gorman <mgorman@suse.de>,
	Minchan Kim <minchan@kernel.org>,
	Lin Feng <linfeng@cn.fujitsu.com>,
	akpm@linux-foundation.org, viro@zeniv.linux.org.uk,
	khlebnikov@openvz.org, walken@google.com,
	kamezawa.hiroyu@jp.fujitsu.com, riel@redhat.com,
	rientjes@google.com, isimatu.yasuaki@jp.fujitsu.com,
	wency@cn.fujitsu.com, laijs@cn.fujitsu.com, jiang.liu@huawei.com,
	jmoyer@redhat.com, linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
	Marek Szyprowski <m.szyprowski@samsung.com>
Subject: Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())
Date: Fri, 17 May 2013 14:30:03 -0400	[thread overview]
Message-ID: <20130517183003.GL1008@kvack.org> (raw)
In-Reply-To: <20130517181708.GG318@lenny.home.zabbo.net>

On Fri, May 17, 2013 at 11:17:08AM -0700, Zach Brown wrote:
> > I ended up working on this a bit today, and managed to cobble together 
> > something that somewhat works -- please see the patch below.
> 
> Just some quick observations:
> 
> > +	ctx->ctx_file = anon_inode_getfile("[aio]", &aio_ctx_fops, ctx, O_RDWR);
> > +	if (IS_ERR(ctx->ctx_file)) {
> > +		ctx->ctx_file = NULL;
> > +		return -EAGAIN;
> > +	}
> 
> It's too bad that aio contexts will now be accounted against the filp
> limits (get_empty_filp -> files_stat.max_files, etc). 

Yeah, that is a downside of this approach.  It would be possible to to 
do it with only an inode/address_space, but that would mean bypassing 
do_mmap(), which is not worth considering.  If it is really an issue, we 
could add a flag to bypass that limit since aio has its own.  
anon_inode_getfile() as it stands is a major problem.

> > +	for (i=0; i<nr_pages; i++) {
> > +		struct page *page;
> > +		void *ptr;
> > +		page = find_or_create_page(ctx->ctx_file->f_inode->i_mapping,
> > +					   i, GFP_KERNEL);
> > +		if (!page) {
> > +			break;
> > +		}
> > +		ptr = kmap(page);
> > +		clear_page(ptr);
> > +		kunmap(page);
> > +		SetPageUptodate(page);
> > +		SetPageDirty(page);
> > +		unlock_page(page);
> > +	}
> 
> If they're GFP_KERNEL then you don't need to kmap them.  But we probably
> want to allocate with GFP_HIGHUSER and then use clear_user_highpage() to
> zero them?

Adding __GFP_ZERO would fix that too. The next respin will include that 
change.  I also have to properly handle the mremap() case as well.

		-ben
-- 
"Thought is the essence of where you are now."

  reply	other threads:[~2013-05-17 18:30 UTC|newest]

Thread overview: 104+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-02-05  9:21 [PATCH V2 0/2] mm: hotplug: implement non-movable version of get_user_pages() to kill long-time pin pages Lin Feng
2013-02-05  9:21 ` Lin Feng
2013-02-05  9:21 ` Lin Feng
2013-02-05  9:21 ` [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable() Lin Feng
2013-02-05  9:21   ` Lin Feng
2013-02-05 12:01   ` Mel Gorman
2013-02-05 12:01     ` Mel Gorman
2013-02-05 12:01     ` Mel Gorman
2013-02-06  0:42     ` Minchan Kim
2013-02-06  0:42       ` Minchan Kim
2013-02-06  0:52       ` Benjamin LaHaise
2013-02-06  0:52         ` Benjamin LaHaise
2013-02-06  0:52         ` Benjamin LaHaise
2013-02-06  9:56       ` Mel Gorman
2013-02-06  9:56         ` Mel Gorman
2013-02-08  2:32         ` Minchan Kim
2013-02-08  2:32           ` Minchan Kim
2013-02-08  2:32           ` Minchan Kim
2013-05-13  9:11         ` Tang Chen
2013-05-13  9:11           ` Tang Chen
2013-05-13  9:11           ` Tang Chen
2013-05-13  9:19           ` Mel Gorman
2013-05-13  9:19             ` Mel Gorman
2013-05-13  9:19             ` Mel Gorman
2013-05-13 14:37             ` Benjamin LaHaise
2013-05-13 14:37               ` Benjamin LaHaise
2013-05-13 14:37               ` Benjamin LaHaise
2013-05-13 14:54               ` Jeff Moyer
2013-05-13 14:54                 ` Jeff Moyer
2013-05-13 14:54                 ` Jeff Moyer
2013-05-13 15:01                 ` Benjamin LaHaise
2013-05-13 15:01                   ` Benjamin LaHaise
2013-05-13 15:01                   ` Benjamin LaHaise
2013-05-14  1:24                   ` Tang Chen
2013-05-14  1:24                     ` Tang Chen
2013-05-14 13:58                     ` Benjamin LaHaise
2013-05-14 13:58                       ` Benjamin LaHaise
2013-05-14 13:58                       ` Benjamin LaHaise
2013-05-14 15:16                       ` chen tang
2013-05-15  2:09                       ` Tang Chen
2013-05-15  2:09                         ` Tang Chen
2013-05-15  7:21                         ` Tang Chen
2013-05-15  7:21                           ` Tang Chen
2013-05-15  7:21                           ` Tang Chen
2013-05-14  3:55             ` Tang Chen
2013-05-14  3:55               ` Tang Chen
2013-05-14  3:55               ` Tang Chen
2013-05-15 13:24               ` Mel Gorman
2013-05-15 13:24                 ` Mel Gorman
2013-05-16  5:54                 ` Tang Chen
2013-05-16  5:54                   ` Tang Chen
2013-05-17  0:23                   ` [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()) Benjamin LaHaise
2013-05-17  0:23                     ` Benjamin LaHaise
2013-05-17  0:23                     ` Benjamin LaHaise
2013-05-17  3:28                     ` Tang Chen
2013-05-17  3:28                       ` Tang Chen
2013-05-17  3:28                       ` Tang Chen
2013-05-17 14:37                       ` Benjamin LaHaise
2013-05-17 14:37                         ` Benjamin LaHaise
2013-05-17 14:37                         ` Benjamin LaHaise
2013-05-21  2:07                         ` Tang Chen
2013-05-21  2:07                           ` Tang Chen
2013-05-21  2:27                           ` Benjamin LaHaise
2013-05-21  2:27                             ` Benjamin LaHaise
2013-05-21  2:27                             ` Benjamin LaHaise
2013-06-11  9:42                             ` Tang Chen
2013-06-11  9:42                               ` Tang Chen
2013-06-11  9:42                               ` Tang Chen
2013-06-11 14:45                               ` Benjamin LaHaise
2013-06-11 14:45                                 ` Benjamin LaHaise
2013-06-28  9:24                                 ` Gu Zheng
2013-06-28  9:24                                   ` Gu Zheng
2013-07-01  7:23                                 ` Gu Zheng
2013-07-01  7:23                                   ` Gu Zheng
2013-07-02 18:00                                   ` Benjamin LaHaise
2013-07-02 18:00                                     ` Benjamin LaHaise
2013-07-02 18:00                                     ` Benjamin LaHaise
2013-07-03  1:53                                     ` Gu Zheng
2013-07-03  1:53                                       ` Gu Zheng
2013-07-03  1:53                                       ` Gu Zheng
2013-07-04  6:51                                     ` Gu Zheng
2013-07-04  6:51                                       ` Gu Zheng
2013-07-04  6:51                                       ` Gu Zheng
2013-07-04 11:41                                       ` Benjamin LaHaise
2013-07-04 11:41                                         ` Benjamin LaHaise
2013-07-04 11:41                                         ` Benjamin LaHaise
2013-07-05  3:21                                         ` Gu Zheng
2013-07-05  3:21                                           ` Gu Zheng
2013-07-05  3:21                                           ` Gu Zheng
2013-05-17 18:17                     ` Zach Brown
2013-05-17 18:17                       ` Zach Brown
2013-05-17 18:30                       ` Benjamin LaHaise [this message]
2013-05-17 18:30                         ` Benjamin LaHaise
2013-05-17 18:30                         ` Benjamin LaHaise
2013-02-20 11:37   ` [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable() Wanpeng Li
2013-02-20 12:39     ` Lin Feng
2013-02-20 12:39       ` Lin Feng
2013-02-20 12:39       ` Lin Feng
2013-02-20 11:37   ` Wanpeng Li
2013-02-20 11:37     ` Wanpeng Li
2013-02-20 11:37   ` Wanpeng Li
2013-02-20 11:37     ` Wanpeng Li
2013-02-05  9:21 ` [PATCH V2 2/2] fs/aio.c: use get_user_pages_non_movable() to pin ring pages when support memory hotremove Lin Feng
2013-02-05  9:21   ` Lin Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130517183003.GL1008@kvack.org \
    --to=bcrl@kvack.org \
    --cc=akpm@linux-foundation.org \
    --cc=isimatu.yasuaki@jp.fujitsu.com \
    --cc=jiang.liu@huawei.com \
    --cc=jmoyer@redhat.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=khlebnikov@openvz.org \
    --cc=laijs@cn.fujitsu.com \
    --cc=linfeng@cn.fujitsu.com \
    --cc=linux-aio@kvack.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=m.szyprowski@samsung.com \
    --cc=mgorman@suse.de \
    --cc=minchan@kernel.org \
    --cc=riel@redhat.com \
    --cc=rientjes@google.com \
    --cc=tangchen@cn.fujitsu.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=walken@google.com \
    --cc=wency@cn.fujitsu.com \
    --cc=zab@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.