[patch -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[patch -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[David Rientjes]

A few hugetlb allocators loop while calling the page allocator and can
potentially prevent rescheduling if the page allocator slowpath is not
utilized.

Conditionally schedule when large numbers of hugepages can be allocated.

Signed-off-by: David Rientjes <rientjes@google.com>
---
 Based on -mm only to prevent merge conflicts with
 "mm/hugetlb.c: warn the user when issues arise on boot due to hugepages"

 mm/hugetlb.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1754,6 +1754,7 @@ static int gather_surplus_pages(struct hstate *h, int delta)
 			break;
 		}
 		list_add(&page->lru, &surplus_list);
+		cond_resched();
 	}
 	allocated += i;
 
@@ -2222,6 +2223,7 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h)
 		} else if (!alloc_fresh_huge_page(h,
 					 &node_states[N_MEMORY]))
 			break;
+		cond_resched();
 	}
 	if (i < h->max_huge_pages) {
 		char buf[32];
@@ -2364,6 +2366,7 @@ static unsigned long set_max_huge_pages(struct hstate *h, unsigned long count,
 			ret = alloc_fresh_gigantic_page(h, nodes_allowed);
 		else
 			ret = alloc_fresh_huge_page(h, nodes_allowed);
+		cond_resched();
 		spin_lock(&hugetlb_lock);
 		if (!ret)
 			goto out;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[Mike Kravetz]

On 06/07/2017 09:03 PM, David Rientjes wrote:
> A few hugetlb allocators loop while calling the page allocator and can
> potentially prevent rescheduling if the page allocator slowpath is not
> utilized.
> 
> Conditionally schedule when large numbers of hugepages can be allocated.
> 
> Signed-off-by: David Rientjes <rientjes@google.com>
> ---
>  Based on -mm only to prevent merge conflicts with
>  "mm/hugetlb.c: warn the user when issues arise on boot due to hugepages"
> 
>  mm/hugetlb.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1754,6 +1754,7 @@ static int gather_surplus_pages(struct hstate *h, int delta)
>  			break;
>  		}
>  		list_add(&page->lru, &surplus_list);
> +		cond_resched();
>  	}
>  	allocated += i;
>  
> @@ -2222,6 +2223,7 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h)
>  		} else if (!alloc_fresh_huge_page(h,
>  					 &node_states[N_MEMORY]))
>  			break;
> +		cond_resched();
>  	}
>  	if (i < h->max_huge_pages) {
>  		char buf[32];
> @@ -2364,6 +2366,7 @@ static unsigned long set_max_huge_pages(struct hstate *h, unsigned long count,
>  			ret = alloc_fresh_gigantic_page(h, nodes_allowed);
>  		else
>  			ret = alloc_fresh_huge_page(h, nodes_allowed);
> +		cond_resched();

Are not the following lines immediately before the above huge page allocation
in set_max_huge_pages, or am I looking at an incorrect version of the file?

		/* yield cpu to avoid soft lockup */
		cond_resched();

-- 
Mike Kravetz

>  		spin_lock(&hugetlb_lock);
>  		if (!ret)
>  			goto out;
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[David Rientjes]

On Wed, 7 Jun 2017, Mike Kravetz wrote:

> > @@ -2364,6 +2366,7 @@ static unsigned long set_max_huge_pages(struct hstate *h, unsigned long count,
> >  			ret = alloc_fresh_gigantic_page(h, nodes_allowed);
> >  		else
> >  			ret = alloc_fresh_huge_page(h, nodes_allowed);
> > +		cond_resched();
> 
> Are not the following lines immediately before the above huge page allocation
> in set_max_huge_pages, or am I looking at an incorrect version of the file?
> 
> 		/* yield cpu to avoid soft lockup */
> 		cond_resched();

Ahh, we don't have this in our tree, thanks for catching it.  The other 
two cond_resched()'s are needed because we have reproduced them, so I'll 
send a v2.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[patch v2 -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[David Rientjes]

A few hugetlb allocators loop while calling the page allocator and can
potentially prevent rescheduling if the page allocator slowpath is not
utilized.

Conditionally schedule when large numbers of hugepages can be allocated.

Signed-off-by: David Rientjes <rientjes@google.com>
---
 Based on -mm only to prevent merge conflicts with
 "mm/hugetlb.c: warn the user when issues arise on boot due to hugepages"

 v2: removed redundant cond_resched() per Mike

 mm/hugetlb.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1754,6 +1754,7 @@ static int gather_surplus_pages(struct hstate *h, int delta)
 			break;
 		}
 		list_add(&page->lru, &surplus_list);
+		cond_resched();
 	}
 	allocated += i;
 
@@ -2222,6 +2223,7 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h)
 		} else if (!alloc_fresh_huge_page(h,
 					 &node_states[N_MEMORY]))
 			break;
+		cond_resched();
 	}
 	if (i < h->max_huge_pages) {
 		char buf[32];

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch v2 -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[Mike Kravetz]

On 06/09/2017 03:36 PM, David Rientjes wrote:
> A few hugetlb allocators loop while calling the page allocator and can
> potentially prevent rescheduling if the page allocator slowpath is not
> utilized.
> 
> Conditionally schedule when large numbers of hugepages can be allocated.
> 
> Signed-off-by: David Rientjes <rientjes@google.com>

Thanks for doing this.

Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>

-- 
Mike Kravetz

> ---
>  Based on -mm only to prevent merge conflicts with
>  "mm/hugetlb.c: warn the user when issues arise on boot due to hugepages"
> 
>  v2: removed redundant cond_resched() per Mike
> 
>  mm/hugetlb.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1754,6 +1754,7 @@ static int gather_surplus_pages(struct hstate *h, int delta)
>  			break;
>  		}
>  		list_add(&page->lru, &surplus_list);
> +		cond_resched();
>  	}
>  	allocated += i;
>  
> @@ -2222,6 +2223,7 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h)
>  		} else if (!alloc_fresh_huge_page(h,
>  					 &node_states[N_MEMORY]))
>  			break;
> +		cond_resched();
>  	}
>  	if (i < h->max_huge_pages) {
>  		char buf[32];
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch v2 -mm] mm, hugetlb: schedule when potentially allocating many hugepages

[Anshuman Khandual]

On 06/10/2017 04:06 AM, David Rientjes wrote:
> A few hugetlb allocators loop while calling the page allocator and can
> potentially prevent rescheduling if the page allocator slowpath is not
> utilized.
> 
> Conditionally schedule when large numbers of hugepages can be allocated.
> 
> Signed-off-by: David Rientjes <rientjes@google.com>

Fixes a task which was getting hung while writing like 10000
hugepages (16MB on POWER8) into /proc/sys/vm/nr_hugepages.

Tested-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>