Re: [PATCH net-next] net: skb: introduce and use a single page frag cache

[Paolo Abeni <pabeni@redhat.com>]

On Wed, 2022-09-21 at 21:33 +0200, Paolo Abeni wrote:
> On Wed, 2022-09-21 at 11:11 -0700, Alexander H Duyck wrote:
[...]
> 
> > >  {
> > >  	struct napi_alloc_cache *nc;
> > >  	struct sk_buff *skb;
> > > +	bool pfmemalloc;
> > >  	void *data;
> > >  
> > >  	DEBUG_NET_WARN_ON_ONCE(!in_softirq());
> > >  	len += NET_SKB_PAD + NET_IP_ALIGN;
> > >  
> > > +	/* When the small frag allocator is available, prefer it over kmalloc
> > > +	 * for small fragments
> > > +	 */
> > > +	if (NAPI_HAS_SMALL_PAGE_FRAG && len <= SKB_WITH_OVERHEAD(1024)) {
> > > +		nc = this_cpu_ptr(&napi_alloc_cache);
> > > +
> > > +		if (sk_memalloc_socks())
> > > +			gfp_mask |= __GFP_MEMALLOC;
> > > +
> > > +		/* we are artificially inflating the allocation size, but
> > > +		 * that is not as bad as it may look like, as:
> > > +		 * - 'len' less then GRO_MAX_HEAD makes little sense
> > > +		 * - larger 'len' values lead to fragment size above 512 bytes
> > > +		 *   as per NAPI_HAS_SMALL_PAGE_FRAG definition
> > > +		 * - kmalloc would use the kmalloc-1k slab for such values
> > > +		 */
> > > +		len = SZ_1K;
> > > +
> > > +		data = page_frag_alloc_1k(&nc->page_small, gfp_mask);
> > > +		pfmemalloc = nc->page_small.pfmemalloc;
> > > +		goto check_data;
> > > +	}
> > > +
> > 
> > It might be better to place this code further down as a branch rather
> > than having to duplicate things up here such as the __GFP_MEMALLOC
> > setting.
> > 
> > You could essentially just put the lines getting the napi_alloc_cache
> > and adding the shared info after the sk_memalloc_socks() check. Then it
> > could just be an if/else block either calling page_frag_alloc or your
> > page_frag_alloc_1k.
> 
> I thought about that option, but I did not like it much because adds a
> conditional in the fast-path for small-size allocation, and the
> duplicate code is very little.
> 
> I can change the code that way, if you have strong opinion in that
> regards.

Thinking again about the above, I now belive that what you suggest is
the right thing to do: my patch ignores the requested 
__GFP_DIRECT_RECLAIM and GFP_DMA flags for small allocation - we always
need to fallback to kmalloc() when the caller ask for them.

TL;DR: I'll move the page_frag_alloc_1k() call below in v2.

Thanks!

Paolo