grow_dev_page's __GFP_MOVABLE

grow_dev_page's __GFP_MOVABLE

[Hugh Dickins]

Hi Mel,

I'm (slightly) worried by your __GFP_MOVABLE in grow_dev_page:
is it valid, given that we come here for filesystem metadata pages
- don't we?  If it is valid, then wouldn't adding __GFP_HIGHMEM
be valid there also?  It'd be very nice to have __GFP_MOVABLE and
__GFP_HIGHMEM on all blockdev pages, but we've concluded in the
past that __GFP_HIGHMEM cannot be allowed without large kmapping
mods throughout the filesystems.  Go back to GFP_NOFS there?

Hugh

--- 2.6.25-rc5/fs/buffer.c	2008-03-05 10:47:40.000000000 +0000
+++ linux/fs/buffer.c	2008-03-11 21:21:10.000000000 +0000
@@ -1029,8 +1029,7 @@ grow_dev_page(struct block_device *bdev,
 	struct page *page;
 	struct buffer_head *bh;
 
-	page = find_or_create_page(inode->i_mapping, index,
-		(mapping_gfp_mask(inode->i_mapping) & ~__GFP_FS)|__GFP_MOVABLE);
+	page = find_or_create_page(inode->i_mapping, index, GFP_NOFS);
 	if (!page)
 		return NULL;
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Mel Gorman]

On (11/03/08 21:33), Hugh Dickins didst pronounce:
> Hi Mel,
> 
> I'm (slightly) worried by your __GFP_MOVABLE in grow_dev_page:
> is it valid, given that we come here for filesystem metadata pages
> - don't we? 

This is a tricky one and the second time it's come up. The pages allocated
here end up on the page cache and had a similar life-cycle to other LRU-pages
in the majority of cases when I checked at the time. The allocations are
labeled MOVABLE, but in this case they can be cleaned and moved to disk
rather than movable by page migration.  Strictly, one would argue that
they could be marked RECLAIMABLE but it increases the number of pageblocks
used by RECLAIMABLE allocations quite considerably and they have a very
different lifecycle which in itself is bad (spreads difficult to reclaim
allocations wider than necessary). Similarly, leaving them GFP_NOFS would
scatter allocations like page table pages wider than expected.

> If it is valid, then wouldn't adding __GFP_HIGHMEM
> be valid there also?  It'd be very nice to have __GFP_MOVABLE and
> __GFP_HIGHMEM on all blockdev pages, but we've concluded in the
> past that __GFP_HIGHMEM cannot be allowed without large kmapping
> mods throughout the filesystems.  Go back to GFP_NOFS there?
> 

I'd prefer not because the current way keeps most LRU pages together,
even if some of them must be allocated from ZONE_NORMAL instead of
ZONE_HIGHMEM but I'm open to being convinced this was a mistake.

Even if we decide to leave this as-is, I should write a patch commenting
on this.

> Hugh
> 
> --- 2.6.25-rc5/fs/buffer.c	2008-03-05 10:47:40.000000000 +0000
> +++ linux/fs/buffer.c	2008-03-11 21:21:10.000000000 +0000
> @@ -1029,8 +1029,7 @@ grow_dev_page(struct block_device *bdev,
>  	struct page *page;
>  	struct buffer_head *bh;
>  
> -	page = find_or_create_page(inode->i_mapping, index,
> -		(mapping_gfp_mask(inode->i_mapping) & ~__GFP_FS)|__GFP_MOVABLE);
> +	page = find_or_create_page(inode->i_mapping, index, GFP_NOFS);
>  	if (!page)
>  		return NULL;
>  
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Hugh Dickins]

On Wed, 12 Mar 2008, Mel Gorman wrote:
> On (11/03/08 21:33), Hugh Dickins didst pronounce:
> > 
> > I'm (slightly) worried by your __GFP_MOVABLE in grow_dev_page:
> > is it valid, given that we come here for filesystem metadata pages
> > - don't we? 
> 
> This is a tricky one and the second time it's come up. The pages allocated
> here end up on the page cache and had a similar life-cycle to other LRU-pages
> in the majority of cases when I checked at the time. The allocations are
> labeled MOVABLE, but in this case they can be cleaned and moved to disk
> rather than movable by page migration.  Strictly, one would argue that
> they could be marked RECLAIMABLE but it increases the number of pageblocks
> used by RECLAIMABLE allocations quite considerably and they have a very
> different lifecycle which in itself is bad (spreads difficult to reclaim
> allocations wider than necessary).

I was finding this ever so hard to understand, but now think I was blocked
by the misapprehension that a filesystem would hold all the metadata pages
associated with an inode in core while that file was open.  That might be
true of some primitive filesystems, but could hardly be true of a grownup
filesystem.  Though even so, I'd expect different kinds of metadata pages
to have significantly different lifecycles, and quite dependent on the
filesystem in question e.g. superblock pages are held in core? inodes?

But you found that the majority are not, their counts merely raised while
being accessed by the filesystem, so made the decision to treat them all
as MOVABLE because most can be reclaimed from the pagecache in the same
way as pagecache pages, which needs significantly less effort than
RECLAIMing from slab.  Too bad about the obstacle to defragmentation
that the held ones would pose.  Okay (if I'm getting it right): you
have to choose one way or the other, you've chosen this way, fine.
And my argument by analogy with __GFP_HIGHMEM was just bogus.

(I guess this is why you added GFP_HIGHUSER_PAGECACHE etc., which to
my mind just obfuscate things further, and intend a patch to remove.)

Though, what prevents them from being genuinely MOVABLE while they're
not transiently in use by the filesystem?  And why does block_dev.c
set mapping gfp_mask to GFP_USER instead of (not yet defined)
GFP_USER_MOVABLE (ah, GFP_USER_PAGECACHE is defined, but unused)? 

> Similarly, leaving them GFP_NOFS would
> scatter allocations like page table pages wider than expected.

Yes, I accept we should do better than GFP_NOFS there: but I'm
now not seeing why it isn't just &~__GFP_FS, with block_dev.c
supplying the MOVABLE.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Mel Gorman]

On (12/03/08 18:11), Hugh Dickins didst pronounce:
> On Wed, 12 Mar 2008, Mel Gorman wrote:
> > On (11/03/08 21:33), Hugh Dickins didst pronounce:
> > > 
> > > I'm (slightly) worried by your __GFP_MOVABLE in grow_dev_page:
> > > is it valid, given that we come here for filesystem metadata pages
> > > - don't we? 
> > 
> > This is a tricky one and the second time it's come up. The pages allocated
> > here end up on the page cache and had a similar life-cycle to other LRU-pages
> > in the majority of cases when I checked at the time. The allocations are
> > labeled MOVABLE, but in this case they can be cleaned and moved to disk
> > rather than movable by page migration.  Strictly, one would argue that
> > they could be marked RECLAIMABLE but it increases the number of pageblocks
> > used by RECLAIMABLE allocations quite considerably and they have a very
> > different lifecycle which in itself is bad (spreads difficult to reclaim
> > allocations wider than necessary).
> 
> I was finding this ever so hard to understand, but now think I was blocked
> by the misapprehension that a filesystem would hold all the metadata pages
> associated with an inode in core while that file was open.  That might be
> true of some primitive filesystems, but could hardly be true of a grownup
> filesystem. 

That is my current understanding.

> Though even so, I'd expect different kinds of metadata pages
> to have significantly different lifecycles, and quite dependent on the
> filesystem in question e.g. superblock pages are held in core? inodes?
> 

This is probably true. To be right, every caller that enters this path should
be updated separetly.

> But you found that the majority are not, their counts merely raised while
> being accessed by the filesystem, so made the decision to treat them all
> as MOVABLE because most can be reclaimed from the pagecache in the same
> way as pagecache pages, which needs significantly less effort than
> RECLAIMing from slab. 

Yes, this is correct. For some filesystems, the pages with buffers can
also be migrated (ext2, ext3, ext4, gfs2, ntfs, ocfs2, xfs) but it's not
universal.

> Too bad about the obstacle to defragmentation
> that the held ones would pose. 

Yeah and I suspect if this is going to hit as a bug report, it will be
related to memory hot-remove. At some point, I'll may have to bite
the bullet and set this place to GFP_NOFS, distinguish between the
different types of caller.

> Okay (if I'm getting it right): you
> have to choose one way or the other, you've chosen this way, fine.
> And my argument by analogy with __GFP_HIGHMEM was just bogus.
> 
> (I guess this is why you added GFP_HIGHUSER_PAGECACHE etc., which to
> my mind just obfuscate things further, and intend a patch to remove.)
> 

Ironically, that was originally introduced to make things easier to
read. 

> Though, what prevents them from being genuinely MOVABLE while they're
> not transiently in use by the filesystem? 

Some of them are. The address_space will have a migratepage() helper if
they are really movable.

>And why does block_dev.c
> set mapping gfp_mask to GFP_USER instead of (not yet defined)
> GFP_USER_MOVABLE (ah, GFP_USER_PAGECACHE is defined, but unused)? 
> 

When I last checked, the blockdev address_space did not implement migratepage()
so pages allocated on its behalf were not movable. I cannot recall if
they ended up on the LRU where they could be reclaimed as normal or not.

> > Similarly, leaving them GFP_NOFS would
> > scatter allocations like page table pages wider than expected.
> 
> Yes, I accept we should do better than GFP_NOFS there: but I'm
> now not seeing why it isn't just &~__GFP_FS, with block_dev.c
> supplying the MOVABLE.
> 

I don't have a quick answer. I've added to the to-do list to revisit
this and see can it be done better.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Badari Pulavarty]

On Thu, 2008-03-13 at 12:07 +0000, Mel Gorman wrote:
> On (12/03/08 18:11), Hugh Dickins didst pronounce:
> > On Wed, 12 Mar 2008, Mel Gorman wrote:
> > > On (11/03/08 21:33), Hugh Dickins didst pronounce:
> > > > 
> > > > I'm (slightly) worried by your __GFP_MOVABLE in grow_dev_page:
> > > > is it valid, given that we come here for filesystem metadata pages
> > > > - don't we? 
> > > 
> > > This is a tricky one and the second time it's come up. The pages allocated
> > > here end up on the page cache and had a similar life-cycle to other LRU-pages
> > > in the majority of cases when I checked at the time. The allocations are
> > > labeled MOVABLE, but in this case they can be cleaned and moved to disk
> > > rather than movable by page migration.  Strictly, one would argue that
> > > they could be marked RECLAIMABLE but it increases the number of pageblocks
> > > used by RECLAIMABLE allocations quite considerably and they have a very
> > > different lifecycle which in itself is bad (spreads difficult to reclaim
> > > allocations wider than necessary).
> > 
> > I was finding this ever so hard to understand, but now think I was blocked
> > by the misapprehension that a filesystem would hold all the metadata pages
> > associated with an inode in core while that file was open.  That might be
> > true of some primitive filesystems, but could hardly be true of a grownup
> > filesystem. 
> 
> That is my current understanding.
> 
> > Though even so, I'd expect different kinds of metadata pages
> > to have significantly different lifecycles, and quite dependent on the
> > filesystem in question e.g. superblock pages are held in core? inodes?
> > 
> 
> This is probably true. To be right, every caller that enters this path should
> be updated separetly.
> 
> > But you found that the majority are not, their counts merely raised while
> > being accessed by the filesystem, so made the decision to treat them all
> > as MOVABLE because most can be reclaimed from the pagecache in the same
> > way as pagecache pages, which needs significantly less effort than
> > RECLAIMing from slab. 
> 
> Yes, this is correct. For some filesystems, the pages with buffers can
> also be migrated (ext2, ext3, ext4, gfs2, ntfs, ocfs2, xfs) but it's not
> universal.
> 
> > Too bad about the obstacle to defragmentation
> > that the held ones would pose. 
> 
> Yeah and I suspect if this is going to hit as a bug report, it will be
> related to memory hot-remove. At some point, I'll may have to bite
> the bullet and set this place to GFP_NOFS, distinguish between the
> different types of caller.
> 
> > Okay (if I'm getting it right): you
> > have to choose one way or the other, you've chosen this way, fine.
> > And my argument by analogy with __GFP_HIGHMEM was just bogus.
> > 
> > (I guess this is why you added GFP_HIGHUSER_PAGECACHE etc., which to
> > my mind just obfuscate things further, and intend a patch to remove.)
> > 
> 
> Ironically, that was originally introduced to make things easier to
> read. 
> 
> > Though, what prevents them from being genuinely MOVABLE while they're
> > not transiently in use by the filesystem? 
> 
> Some of them are. The address_space will have a migratepage() helper if
> they are really movable.
> 
> >And why does block_dev.c
> > set mapping gfp_mask to GFP_USER instead of (not yet defined)
> > GFP_USER_MOVABLE (ah, GFP_USER_PAGECACHE is defined, but unused)? 
> > 
> 
> When I last checked, the blockdev address_space did not implement migratepage()
> so pages allocated on its behalf were not movable. I cannot recall if
> they ended up on the LRU where they could be reclaimed as normal or not.
> 
> > > Similarly, leaving them GFP_NOFS would
> > > scatter allocations like page table pages wider than expected.
> > 
> > Yes, I accept we should do better than GFP_NOFS there: but I'm
> > now not seeing why it isn't just &~__GFP_FS, with block_dev.c
> > supplying the MOVABLE.
> > 
> 
> I don't have a quick answer. I've added to the to-do list to revisit
> this and see can it be done better.

Mel,

All I can say is, marking grow_dev_page() __GFP_MOVABLE is causing
nothing but trouble in my hotplug memory remove testing :(

I constantly see that even though memblock is marked "removable", I
can't move the allocations. Most of the times these allocations came
from grow_dev_pages or its friends :(

Either these pages are not movable/migratable or code is not working
or filesystem/block device is holding them up :(


memory offlining 0x8000 to 0x9000 failed

page_owner shows:

Page allocated via order 0, mask 0x120050
PFN 30625 Block 7 type 2          Flags      L
[0xc0000000000c511c] .alloc_pages_current+208
[0xc0000000001049d8] .__find_get_block_slow+88
[0xc0000000004f0bbc] .__wait_on_bit+232
[0xc0000000000994ec] .__page_cache_alloc+24
[0xc000000000104fd8] .__find_get_block+272
[0xc00000000009a124] .find_or_create_page+76
[0xc0000000001063fc] .unlock_buffer+48
[0xc000000000105280] .__getblk+312


Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Mel Gorman]

On (13/03/08 07:05), Badari Pulavarty didst pronounce:
> > <SNIP>
> 
> Mel,
> 
> All I can say is, marking grow_dev_page() __GFP_MOVABLE is causing
> nothing but trouble in my hotplug memory remove testing :(
> 

Dirt.

> I constantly see that even though memblock is marked "removable", I
> can't move the allocations. Most of the times these allocations came
> from grow_dev_pages or its friends :(
> 
> Either these pages are not movable/migratable or code is not working
> or filesystem/block device is holding them up :(
> 

Or no effort is being made to reclaim pages whose address_space has no
migratepages() handler.

> 
> memory offlining 0x8000 to 0x9000 failed
> 
> page_owner shows:
> 
> Page allocated via order 0, mask 0x120050
> PFN 30625 Block 7 type 2          Flags      L

This page is indicated as being on the LRU so it should have been possible
to reclaim. Is memory hot-remove making any effort to reclaim this page or
is it depending only on page migration?

> [0xc0000000000c511c] .alloc_pages_current+208
> [0xc0000000001049d8] .__find_get_block_slow+88
> [0xc0000000004f0bbc] .__wait_on_bit+232
> [0xc0000000000994ec] .__page_cache_alloc+24
> [0xc000000000104fd8] .__find_get_block+272
> [0xc00000000009a124] .find_or_create_page+76
> [0xc0000000001063fc] .unlock_buffer+48
> [0xc000000000105280] .__getblk+312
> 
> 
> Thanks,
> Badari
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Badari Pulavarty]

On Thu, 2008-03-13 at 15:44 +0000, Mel Gorman wrote:
> On (13/03/08 07:05), Badari Pulavarty didst pronounce:
> > > <SNIP>
> > 
> > Mel,
> > 
> > All I can say is, marking grow_dev_page() __GFP_MOVABLE is causing
> > nothing but trouble in my hotplug memory remove testing :(
> > 
> 
> Dirt.
> 
> > I constantly see that even though memblock is marked "removable", I
> > can't move the allocations. Most of the times these allocations came
> > from grow_dev_pages or its friends :(
> > 
> > Either these pages are not movable/migratable or code is not working
> > or filesystem/block device is holding them up :(
> > 
> 
> Or no effort is being made to reclaim pages whose address_space has no
> migratepages() handler.
> 
> > 
> > memory offlining 0x8000 to 0x9000 failed
> > 
> > page_owner shows:
> > 
> > Page allocated via order 0, mask 0x120050
> > PFN 30625 Block 7 type 2          Flags      L
> 
> This page is indicated as being on the LRU so it should have been possible
> to reclaim. Is memory hot-remove making any effort to reclaim this page or
> is it depending only on page migration?

offline_pages() finds all the pages on LRU and tries to migrate them by
calling unmap_and_move(). I don't see any explicit attempt to reclaim.
It tries to migrate the page (move_to_new_page()), but what I have seen
in the past is that these pages have buffer heads attached to them. 
So, migrate_page_move_mapping() fails to release the page. (BTW,
I narrowed this in Oct 2007 and forgot most of the details). I can
take a closer look again. Can we reclaim these pages easily ?

> 
> > [0xc0000000000c511c] .alloc_pages_current+208
> > [0xc0000000001049d8] .__find_get_block_slow+88
> > [0xc0000000004f0bbc] .__wait_on_bit+232
> > [0xc0000000000994ec] .__page_cache_alloc+24
> > [0xc000000000104fd8] .__find_get_block+272
> > [0xc00000000009a124] .find_or_create_page+76
> > [0xc0000000001063fc] .unlock_buffer+48
> > [0xc000000000105280] .__getblk+312
> > 

Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Mel Gorman]

On (13/03/08 16:50), Badari Pulavarty didst pronounce:
> > > <SNIP>
> > > page_owner shows:
> > > 
> > > Page allocated via order 0, mask 0x120050
> > > PFN 30625 Block 7 type 2          Flags      L
> > 
> > This page is indicated as being on the LRU so it should have been possible
> > to reclaim. Is memory hot-remove making any effort to reclaim this page or
> > is it depending only on page migration?
> 
> offline_pages() finds all the pages on LRU and tries to migrate them by
> calling unmap_and_move(). I don't see any explicit attempt to reclaim.
> It tries to migrate the page (move_to_new_page()), but what I have seen
> in the past is that these pages have buffer heads attached to them. 
> So, migrate_page_move_mapping() fails to release the page. (BTW,
> I narrowed this in Oct 2007 and forgot most of the details). I can
> take a closer look again. Can we reclaim these pages easily ?
> 

They should be, or huge page allocations using lumpy reclaim would also
be failing all the time.

> > 
> > > [0xc0000000000c511c] .alloc_pages_current+208
> > > [0xc0000000001049d8] .__find_get_block_slow+88
> > > [0xc0000000004f0bbc] .__wait_on_bit+232
> > > [0xc0000000000994ec] .__page_cache_alloc+24
> > > [0xc000000000104fd8] .__find_get_block+272
> > > [0xc00000000009a124] .find_or_create_page+76
> > > [0xc0000000001063fc] .unlock_buffer+48
> > > [0xc000000000105280] .__getblk+312
> > > 
> 
> Thanks,
> Badari
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Badari Pulavarty]

On Fri, 2008-03-14 at 11:47 +0000, Mel Gorman wrote:
> On (13/03/08 16:50), Badari Pulavarty didst pronounce:
> > > > <SNIP>
> > > > page_owner shows:
> > > > 
> > > > Page allocated via order 0, mask 0x120050
> > > > PFN 30625 Block 7 type 2          Flags      L
> > > 
> > > This page is indicated as being on the LRU so it should have been possible
> > > to reclaim. Is memory hot-remove making any effort to reclaim this page or
> > > is it depending only on page migration?
> > 
> > offline_pages() finds all the pages on LRU and tries to migrate them by
> > calling unmap_and_move(). I don't see any explicit attempt to reclaim.
> > It tries to migrate the page (move_to_new_page()), but what I have seen
> > in the past is that these pages have buffer heads attached to them. 
> > So, migrate_page_move_mapping() fails to release the page. (BTW,
> > I narrowed this in Oct 2007 and forgot most of the details). I can
> > take a closer look again. Can we reclaim these pages easily ?
> > 
> 
> They should be, or huge page allocations using lumpy reclaim would also
> be failing all the time.

Hi Mel,

These pages are on LRU and clean. In order to reclaim these pages
(looking at pageout()), all we need to do is try_to_release_page().
fallback_migrate_page() does this but fails to free it up. What
else I can do here to force reclaim these ?

Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Badari Pulavarty]

On Fri, 2008-03-14 at 08:05 -0800, Badari Pulavarty wrote:
> On Fri, 2008-03-14 at 11:47 +0000, Mel Gorman wrote:
> > On (13/03/08 16:50), Badari Pulavarty didst pronounce:
> > > > > <SNIP>
> > > > > page_owner shows:
> > > > > 
> > > > > Page allocated via order 0, mask 0x120050
> > > > > PFN 30625 Block 7 type 2          Flags      L
> > > > 
> > > > This page is indicated as being on the LRU so it should have been possible
> > > > to reclaim. Is memory hot-remove making any effort to reclaim this page or
> > > > is it depending only on page migration?
> > > 
> > > offline_pages() finds all the pages on LRU and tries to migrate them by
> > > calling unmap_and_move(). I don't see any explicit attempt to reclaim.
> > > It tries to migrate the page (move_to_new_page()), but what I have seen
> > > in the past is that these pages have buffer heads attached to them. 
> > > So, migrate_page_move_mapping() fails to release the page. (BTW,
> > > I narrowed this in Oct 2007 and forgot most of the details). I can
> > > take a closer look again. Can we reclaim these pages easily ?
> > > 
> > 
> > They should be, or huge page allocations using lumpy reclaim would also
> > be failing all the time.
> 
> Hi Mel,
> 
> These pages are on LRU and clean. In order to reclaim these pages
> (looking at pageout()), all we need to do is try_to_release_page().
> fallback_migrate_page() does this but fails to free it up. What
> else I can do here to force reclaim these ?

In other words, caller has a ref on the buffer_head by calling
getblk(). So, try_to_release_page() would fail since buffer_busy().
These buffers could be holding meta-data (super block, bitmaps etc.)
for a file system. Till fs gets rid of the ref, there is nothing
much we can do. Isn't it ?



Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Andy Whitcroft]

On Fri, Mar 14, 2008 at 10:52:45AM -0800, Badari Pulavarty wrote:
> On Fri, 2008-03-14 at 08:05 -0800, Badari Pulavarty wrote:
> > On Fri, 2008-03-14 at 11:47 +0000, Mel Gorman wrote:
> > > On (13/03/08 16:50), Badari Pulavarty didst pronounce:
> > > > > > <SNIP>
> > > > > > page_owner shows:
> > > > > > 
> > > > > > Page allocated via order 0, mask 0x120050
> > > > > > PFN 30625 Block 7 type 2          Flags      L
> > > > > 
> > > > > This page is indicated as being on the LRU so it should have been possible
> > > > > to reclaim. Is memory hot-remove making any effort to reclaim this page or
> > > > > is it depending only on page migration?
> > > > 
> > > > offline_pages() finds all the pages on LRU and tries to migrate them by
> > > > calling unmap_and_move(). I don't see any explicit attempt to reclaim.
> > > > It tries to migrate the page (move_to_new_page()), but what I have seen
> > > > in the past is that these pages have buffer heads attached to them. 
> > > > So, migrate_page_move_mapping() fails to release the page. (BTW,
> > > > I narrowed this in Oct 2007 and forgot most of the details). I can
> > > > take a closer look again. Can we reclaim these pages easily ?
> > > > 
> > > 
> > > They should be, or huge page allocations using lumpy reclaim would also
> > > be failing all the time.
> > 
> > Hi Mel,
> > 
> > These pages are on LRU and clean. In order to reclaim these pages
> > (looking at pageout()), all we need to do is try_to_release_page().
> > fallback_migrate_page() does this but fails to free it up. What
> > else I can do here to force reclaim these ?
> 
> In other words, caller has a ref on the buffer_head by calling
> getblk(). So, try_to_release_page() would fail since buffer_busy().
> These buffers could be holding meta-data (super block, bitmaps etc.)
> for a file system. Till fs gets rid of the ref, there is nothing
> much we can do. Isn't it ?

Cirtainly when fragmentation avoidance was young we did see a lot of
this as some of the meta-data was badly marked.  What filesystem is this
running on?

-apw

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Badari Pulavarty]

Andy Whitcroft wrote:
> On Fri, Mar 14, 2008 at 10:52:45AM -0800, Badari Pulavarty wrote:
>   
>> On Fri, 2008-03-14 at 08:05 -0800, Badari Pulavarty wrote:
>>     
>>> On Fri, 2008-03-14 at 11:47 +0000, Mel Gorman wrote:
>>>       
>>>> On (13/03/08 16:50), Badari Pulavarty didst pronounce:
>>>>         
>>>>>>> <SNIP>
>>>>>>> page_owner shows:
>>>>>>>
>>>>>>> Page allocated via order 0, mask 0x120050
>>>>>>> PFN 30625 Block 7 type 2          Flags      L
>>>>>>>               
>>>>>> This page is indicated as being on the LRU so it should have been possible
>>>>>> to reclaim. Is memory hot-remove making any effort to reclaim this page or
>>>>>> is it depending only on page migration?
>>>>>>             
>>>>> offline_pages() finds all the pages on LRU and tries to migrate them by
>>>>> calling unmap_and_move(). I don't see any explicit attempt to reclaim.
>>>>> It tries to migrate the page (move_to_new_page()), but what I have seen
>>>>> in the past is that these pages have buffer heads attached to them. 
>>>>> So, migrate_page_move_mapping() fails to release the page. (BTW,
>>>>> I narrowed this in Oct 2007 and forgot most of the details). I can
>>>>> take a closer look again. Can we reclaim these pages easily ?
>>>>>
>>>>>           
>>>> They should be, or huge page allocations using lumpy reclaim would also
>>>> be failing all the time.
>>>>         
>>> Hi Mel,
>>>
>>> These pages are on LRU and clean. In order to reclaim these pages
>>> (looking at pageout()), all we need to do is try_to_release_page().
>>> fallback_migrate_page() does this but fails to free it up. What
>>> else I can do here to force reclaim these ?
>>>       
>> In other words, caller has a ref on the buffer_head by calling
>> getblk(). So, try_to_release_page() would fail since buffer_busy().
>> These buffers could be holding meta-data (super block, bitmaps etc.)
>> for a file system. Till fs gets rid of the ref, there is nothing
>> much we can do. Isn't it ?
>>     
>
> Cirtainly when fragmentation avoidance was young we did see a lot of
> this as some of the meta-data was badly marked.  What filesystem is this
> running on?
>
> -apw
>   
I have seen this with reiserfs and ext3.

Thanks,
Badari

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: grow_dev_page's __GFP_MOVABLE

[Mel Gorman]

On (14/03/08 08:05), Badari Pulavarty didst pronounce:
> On Fri, 2008-03-14 at 11:47 +0000, Mel Gorman wrote:
> > On (13/03/08 16:50), Badari Pulavarty didst pronounce:
> > > > > <SNIP>
> > > > > page_owner shows:
> > > > > 
> > > > > Page allocated via order 0, mask 0x120050
> > > > > PFN 30625 Block 7 type 2          Flags      L
> > > > 
> > > > This page is indicated as being on the LRU so it should have been possible
> > > > to reclaim. Is memory hot-remove making any effort to reclaim this page or
> > > > is it depending only on page migration?
> > > 
> > > offline_pages() finds all the pages on LRU and tries to migrate them by
> > > calling unmap_and_move(). I don't see any explicit attempt to reclaim.
> > > It tries to migrate the page (move_to_new_page()), but what I have seen
> > > in the past is that these pages have buffer heads attached to them. 
> > > So, migrate_page_move_mapping() fails to release the page. (BTW,
> > > I narrowed this in Oct 2007 and forgot most of the details). I can
> > > take a closer look again. Can we reclaim these pages easily ?
> > > 
> > 
> > They should be, or huge page allocations using lumpy reclaim would also
> > be failing all the time.
> 
> Hi Mel,
> 
> These pages are on LRU and clean. In order to reclaim these pages
> (looking at pageout()), all we need to do is try_to_release_page().
> fallback_migrate_page() does this but fails to free it up. What
> else I can do here to force reclaim these ?
> 

The pageout path is involved (look at shrink_list() and pageout() in
combination) and calling just try_to_release_page() may not be enough (may
have to be cleaned first, unmapped etc).

After attempting to migrate a page, it would seem reasonable to isolate pages
that cannot migrate but are on the LRU similar to what isolate_lru_pages()
does (see the switch statement)[1]. Put them on a private list and call
shrink_page_list() with PAGEOUT_IO_SYNC so that it will wait for IO to
complete on any dirty pages. That will take all the steps normal reclaim
does.

[1] Don't forget you need to hold with spin_lock_irq the zone->lru_lock

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>