[PATCH] mlx4_en: don't wait for high order page allocation

[PATCH] mlx4_en: don't wait for high order page allocation

[Shaohua Li]

High order page allocation can cause direct memory compaction and harm
performance. The patch makes the high order page allocation don't wait,
so not trigger direct memory compaction with memory pressure. More
details can be found in a similar patch for net core:
http://marc.info/?l=linux-mm&m=143406665720428&w=2

Cc: Amir Vadai <amirv@mellanox.com>
Cc: Ido Shamay <idos@mellanox.com>
Cc: Eric Dumazet <edumazet@google.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 2a77a6b..9bc4143 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -60,8 +60,11 @@ static int mlx4_alloc_pages(struct mlx4_en_priv *priv,
 	for (order = MLX4_EN_ALLOC_PREFER_ORDER; ;) {
 		gfp_t gfp = _gfp;
 
-		if (order)
+		if (order) {
+			if ((PAGE_SIZE << (order - 1)) >= frag_info->frag_size)
+				gfp &= ~__GFP_WAIT;
 			gfp |= __GFP_COMP | __GFP_NOWARN;
+		}
 		page = alloc_pages(gfp, order);
 		if (likely(page))
 			break;
-- 
1.8.1

Re: [PATCH] mlx4_en: don't wait for high order page allocation

[Alexander Duyck]

On 06/12/2015 09:50 AM, Shaohua Li wrote:
> High order page allocation can cause direct memory compaction and harm
> performance. The patch makes the high order page allocation don't wait,
> so not trigger direct memory compaction with memory pressure. More
> details can be found in a similar patch for net core:
> http://marc.info/?l=linux-mm&m=143406665720428&w=2
>
> Cc: Amir Vadai <amirv@mellanox.com>
> Cc: Ido Shamay <idos@mellanox.com>
> Cc: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>   drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> index 2a77a6b..9bc4143 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> @@ -60,8 +60,11 @@ static int mlx4_alloc_pages(struct mlx4_en_priv *priv,
>   	for (order = MLX4_EN_ALLOC_PREFER_ORDER; ;) {
>   		gfp_t gfp = _gfp;
>   
> -		if (order)
> +		if (order) {
> +			if ((PAGE_SIZE << (order - 1)) >= frag_info->frag_size)
> +				gfp &= ~__GFP_WAIT;
>   			gfp |= __GFP_COMP | __GFP_NOWARN;
> +		}
>   		page = alloc_pages(gfp, order);
>   		if (likely(page))
>   			break;

Is this even really necessary?  I would thing the fact that the refill 
is done using GFP_ATOMIC would be enough to cover the frequently used 
cases.  I wouldn't think the initial allocation when the interface is 
brought up would be something that is a big enough deal to justify being 
fixed in this case.

- Alex

Re: [PATCH] mlx4_en: don't wait for high order page allocation

[Shaohua Li]

On Fri, Jun 12, 2015 at 10:05:42AM -0700, Alexander Duyck wrote:
> On 06/12/2015 09:50 AM, Shaohua Li wrote:
> >High order page allocation can cause direct memory compaction and harm
> >performance. The patch makes the high order page allocation don't wait,
> >so not trigger direct memory compaction with memory pressure. More
> >details can be found in a similar patch for net core:
> >http://marc.info/?l=linux-mm&m=143406665720428&w=2
> >
> >Cc: Amir Vadai <amirv@mellanox.com>
> >Cc: Ido Shamay <idos@mellanox.com>
> >Cc: Eric Dumazet <edumazet@google.com>
> >Signed-off-by: Shaohua Li <shli@fb.com>
> >---
> >  drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> >diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> >index 2a77a6b..9bc4143 100644
> >--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> >+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> >@@ -60,8 +60,11 @@ static int mlx4_alloc_pages(struct mlx4_en_priv *priv,
> >  	for (order = MLX4_EN_ALLOC_PREFER_ORDER; ;) {
> >  		gfp_t gfp = _gfp;
> >-		if (order)
> >+		if (order) {
> >+			if ((PAGE_SIZE << (order - 1)) >= frag_info->frag_size)
> >+				gfp &= ~__GFP_WAIT;
> >  			gfp |= __GFP_COMP | __GFP_NOWARN;
> >+		}
> >  		page = alloc_pages(gfp, order);
> >  		if (likely(page))
> >  			break;
> 
> Is this even really necessary?  I would thing the fact that the
> refill is done using GFP_ATOMIC would be enough to cover the
> frequently used cases.  I wouldn't think the initial allocation when
> the interface is brought up would be something that is a big enough
> deal to justify being fixed in this case.

Ok, if the allocation is always using GFP_ATOMIC at runtime, we
don't need this of course. please ignore it then.

Thanks,
Shaohua