Re: [PATCH] net: Update netdev_alloc_frag to work more efficiently with TCP and GRO

[Alexander Duyck <alexander.duyck@gmail.com>]

On 6/20/2012 6:21 AM, Eric Dumazet wrote:
> On Wed, 2012-06-20 at 10:17 +0200, Eric Dumazet wrote:
>
>> Strange, I did again benchs with order-2 allocations and got good
>> results this time, but with latest net-next, maybe things have changed
>> since last time I did this.
>>
>> (netdev_alloc_frag(), get_page_from_freelist() and put_page() less
>> prevalent in perf results)
>>
> In fact, since SLUB uses order-3 for kmalloc-2048, I felt lucky to try
> this as well, and results are really good, on ixgbe at least.
>
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index 5b21522..ffd2cba 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -299,6 +299,9 @@ struct netdev_alloc_cache {
>   };
>   static DEFINE_PER_CPU(struct netdev_alloc_cache, netdev_alloc_cache);
>
> +#define MAX_NETDEV_FRAGSIZE	max_t(unsigned int, PAGE_SIZE, 32768)
> +#define NETDEV_FRAG_ORDER	get_order(MAX_NETDEV_FRAGSIZE)
> +
>   /**
>    * netdev_alloc_frag - allocate a page fragment
>    * @fragsz: fragment size
> @@ -316,11 +319,13 @@ void *netdev_alloc_frag(unsigned int fragsz)
>   	nc =&__get_cpu_var(netdev_alloc_cache);
>   	if (unlikely(!nc->page)) {
>   refill:
> -		nc->page = alloc_page(GFP_ATOMIC | __GFP_COLD);
> +		nc->page = alloc_pages(GFP_ATOMIC | __GFP_COLD |
> +				       (NETDEV_FRAG_ORDER ? __GFP_COMP : 0),
> +				       NETDEV_FRAG_ORDER);
>   		nc->offset = 0;
>   	}
I was wondering if you needed the check for NETDEV_FRAG_ORDER here.  
 From what I can tell setting __GFP_COMP for an order 0 page has no 
effect since it only seems to get checked in prep_new_page and that is 
after a check to verify if the page is order 0 or not.

Thanks,

Alex