[PATCH 1/2] mm: migrate: consolidate mem_cgroup_migrate() calls

[PATCH 1/2] mm: migrate: consolidate mem_cgroup_migrate() calls

[Johannes Weiner]

Rather than scattering mem_cgroup_migrate() calls all over the place,
have a single call from a safe place where every migration operation
eventually ends up in - migrate_page_copy().

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Hugh Dickins <hughd@google.com>
---
 mm/migrate.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 17db63b2dd36..90cbf7c65cac 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -331,8 +331,6 @@ int migrate_page_move_mapping(struct address_space *mapping,
 		if (PageSwapBacked(page))
 			SetPageSwapBacked(newpage);
 
-		mem_cgroup_migrate(page, newpage);
-
 		return MIGRATEPAGE_SUCCESS;
 	}
 
@@ -428,8 +426,6 @@ int migrate_page_move_mapping(struct address_space *mapping,
 	}
 	local_irq_enable();
 
-	mem_cgroup_migrate(page, newpage);
-
 	return MIGRATEPAGE_SUCCESS;
 }
 
@@ -471,8 +467,6 @@ int migrate_huge_page_move_mapping(struct address_space *mapping,
 
 	spin_unlock_irq(&mapping->tree_lock);
 
-	mem_cgroup_migrate(page, newpage);
-
 	return MIGRATEPAGE_SUCCESS;
 }
 
@@ -586,6 +580,8 @@ void migrate_page_copy(struct page *newpage, struct page *page)
 		end_page_writeback(newpage);
 
 	copy_page_owner(page, newpage);
+
+	mem_cgroup_migrate(page, newpage);
 }
 
 /************************************************************
@@ -1846,7 +1842,6 @@ fail_putback:
 	}
 
 	mlock_migrate_page(new_page, page);
-	mem_cgroup_migrate(page, new_page);
 	page_remove_rmap(page, true);
 	set_page_owner_migrate_reason(new_page, MR_NUMA_MISPLACED);
 
-- 
2.7.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/2] mm: memcontrol: drop unnecessary lru locking from mem_cgroup_migrate()

[Johannes Weiner]

Migration accounting in the memory controller used to have to handle
both oldpage and newpage being on the LRU already; fuse's page cache
replacement used to pass a recycled newpage that had been uncharged
but not freed and removed from the LRU, and the memcg migration code
used to uncharge oldpage to "pass on" the existing charge to newpage.

Nowadays, pages are no longer uncharged when truncated from the page
cache, but rather only at free time, so if a LRU page is recycled in
page cache replacement it'll also still be charged. And we bail out of
the charge transfer altogether in that case. Tell commit_charge() that
we know newpage is not on the LRU, to avoid taking the zone->lru_lock
unnecessarily from the migration path.

But also, oldpage is no longer uncharged inside migration. We only use
oldpage for its page->mem_cgroup and page size, so we don't care about
its LRU state anymore either. Remove any mention from the kernel doc.

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Hugh Dickins <hughd@google.com>
---
 mm/memcontrol.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 3e4199830456..42882c1e7fce 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5489,7 +5489,6 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
  * be uncharged upon free.
  *
  * Both pages must be locked, @newpage->mapping must be set up.
- * Either or both pages might be on the LRU already.
  */
 void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 {
@@ -5524,7 +5523,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 		page_counter_charge(&memcg->memsw, nr_pages);
 	css_get_many(&memcg->css, nr_pages);
 
-	commit_charge(newpage, memcg, true);
+	commit_charge(newpage, memcg, false);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, newpage, compound, nr_pages);
-- 
2.7.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/2] mm: memcontrol: drop unnecessary lru locking from mem_cgroup_migrate()

[Vladimir Davydov]

On Thu, Feb 04, 2016 at 03:07:47PM -0500, Johannes Weiner wrote:
> Migration accounting in the memory controller used to have to handle
> both oldpage and newpage being on the LRU already; fuse's page cache
> replacement used to pass a recycled newpage that had been uncharged
> but not freed and removed from the LRU, and the memcg migration code
> used to uncharge oldpage to "pass on" the existing charge to newpage.
> 
> Nowadays, pages are no longer uncharged when truncated from the page
> cache, but rather only at free time, so if a LRU page is recycled in
> page cache replacement it'll also still be charged. And we bail out of
> the charge transfer altogether in that case. Tell commit_charge() that
> we know newpage is not on the LRU, to avoid taking the zone->lru_lock
> unnecessarily from the migration path.
> 
> But also, oldpage is no longer uncharged inside migration. We only use
> oldpage for its page->mem_cgroup and page size, so we don't care about
> its LRU state anymore either. Remove any mention from the kernel doc.
> 
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Hugh Dickins <hughd@google.com>

Acked-by: Vladimir Davydov <vdavydov@virtuozzo.com>

Nit:

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ae8b81c55685..120118f3ce0a 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5483,6 +5483,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	unsigned int nr_pages;
 	bool compound;
 
+	VM_BUG_ON_PAGE(PageLRU(newpage), newpage);
 	VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
 	VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);
 	VM_BUG_ON_PAGE(PageAnon(oldpage) != PageAnon(newpage), newpage);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/2] mm: memcontrol: drop unnecessary lru locking from mem_cgroup_migrate()

[Johannes Weiner]

> On Feb 7, 2016, at 1:41 PM, Vladimir Davydov <vdavydov@virtuozzo.com> wrote:
> 
>> On Thu, Feb 04, 2016 at 03:07:47PM -0500, Johannes Weiner wrote:
>> Migration accounting in the memory controller used to have to handle
>> both oldpage and newpage being on the LRU already; fuse's page cache
>> replacement used to pass a recycled newpage that had been uncharged
>> but not freed and removed from the LRU, and the memcg migration code
>> used to uncharge oldpage to "pass on" the existing charge to newpage.
>> 
>> Nowadays, pages are no longer uncharged when truncated from the page
>> cache, but rather only at free time, so if a LRU page is recycled in
>> page cache replacement it'll also still be charged. And we bail out of
>> the charge transfer altogether in that case. Tell commit_charge() that
>> we know newpage is not on the LRU, to avoid taking the zone->lru_lock
>> unnecessarily from the migration path.
>> 
>> But also, oldpage is no longer uncharged inside migration. We only use
>> oldpage for its page->mem_cgroup and page size, so we don't care about
>> its LRU state anymore either. Remove any mention from the kernel doc.
>> 
>> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
>> Suggested-by: Hugh Dickins <hughd@google.com>
> 
> Acked-by: Vladimir Davydov <vdavydov@virtuozzo.com>

Thanks!

> @@ -5483,6 +5483,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
>    unsigned int nr_pages;
>    bool compound;
> 
> +    VM_BUG_ON_PAGE(PageLRU(newpage), newpage);

That's actually possible for fuse. But in that case newpage is charged and we bail.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/2] mm: memcontrol: drop unnecessary lru locking from mem_cgroup_migrate()

[Michal Hocko]

On Thu 04-02-16 15:07:47, Johannes Weiner wrote:
> Migration accounting in the memory controller used to have to handle
> both oldpage and newpage being on the LRU already; fuse's page cache
> replacement used to pass a recycled newpage that had been uncharged
> but not freed and removed from the LRU, and the memcg migration code
> used to uncharge oldpage to "pass on" the existing charge to newpage.
> 
> Nowadays, pages are no longer uncharged when truncated from the page
> cache, but rather only at free time, so if a LRU page is recycled in
> page cache replacement it'll also still be charged. And we bail out of
> the charge transfer altogether in that case. Tell commit_charge() that
> we know newpage is not on the LRU, to avoid taking the zone->lru_lock
> unnecessarily from the migration path.
> 
> But also, oldpage is no longer uncharged inside migration. We only use
> oldpage for its page->mem_cgroup and page size, so we don't care about
> its LRU state anymore either. Remove any mention from the kernel doc.
> 
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Hugh Dickins <hughd@google.com>

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  mm/memcontrol.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 3e4199830456..42882c1e7fce 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -5489,7 +5489,6 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
>   * be uncharged upon free.
>   *
>   * Both pages must be locked, @newpage->mapping must be set up.
> - * Either or both pages might be on the LRU already.
>   */
>  void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
>  {
> @@ -5524,7 +5523,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
>  		page_counter_charge(&memcg->memsw, nr_pages);
>  	css_get_many(&memcg->css, nr_pages);
>  
> -	commit_charge(newpage, memcg, true);
> +	commit_charge(newpage, memcg, false);
>  
>  	local_irq_disable();
>  	mem_cgroup_charge_statistics(memcg, newpage, compound, nr_pages);
> -- 
> 2.7.0

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] mm: migrate: consolidate mem_cgroup_migrate() calls

[Vladimir Davydov]

On Thu, Feb 04, 2016 at 03:07:46PM -0500, Johannes Weiner wrote:
> Rather than scattering mem_cgroup_migrate() calls all over the place,
> have a single call from a safe place where every migration operation
> eventually ends up in - migrate_page_copy().
> 
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Hugh Dickins <hughd@google.com>

Acked-by: Vladimir Davydov <vdavydov@virtuozzo.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] mm: migrate: consolidate mem_cgroup_migrate() calls

[Michal Hocko]

On Thu 04-02-16 15:07:46, Johannes Weiner wrote:
> Rather than scattering mem_cgroup_migrate() calls all over the place,
> have a single call from a safe place where every migration operation
> eventually ends up in - migrate_page_copy().

This is very nice!

> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Hugh Dickins <hughd@google.com>

Sorry for coming so late.
Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  mm/migrate.c | 9 ++-------
>  1 file changed, 2 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 17db63b2dd36..90cbf7c65cac 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -331,8 +331,6 @@ int migrate_page_move_mapping(struct address_space *mapping,
>  		if (PageSwapBacked(page))
>  			SetPageSwapBacked(newpage);
>  
> -		mem_cgroup_migrate(page, newpage);
> -
>  		return MIGRATEPAGE_SUCCESS;
>  	}
>  
> @@ -428,8 +426,6 @@ int migrate_page_move_mapping(struct address_space *mapping,
>  	}
>  	local_irq_enable();
>  
> -	mem_cgroup_migrate(page, newpage);
> -
>  	return MIGRATEPAGE_SUCCESS;
>  }
>  
> @@ -471,8 +467,6 @@ int migrate_huge_page_move_mapping(struct address_space *mapping,
>  
>  	spin_unlock_irq(&mapping->tree_lock);
>  
> -	mem_cgroup_migrate(page, newpage);
> -
>  	return MIGRATEPAGE_SUCCESS;
>  }
>  
> @@ -586,6 +580,8 @@ void migrate_page_copy(struct page *newpage, struct page *page)
>  		end_page_writeback(newpage);
>  
>  	copy_page_owner(page, newpage);
> +
> +	mem_cgroup_migrate(page, newpage);
>  }
>  
>  /************************************************************
> @@ -1846,7 +1842,6 @@ fail_putback:
>  	}
>  
>  	mlock_migrate_page(new_page, page);
> -	mem_cgroup_migrate(page, new_page);
>  	page_remove_rmap(page, true);
>  	set_page_owner_migrate_reason(new_page, MR_NUMA_MISPLACED);
>  
> -- 
> 2.7.0

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>