Re: [PATCH] slub: Avoid irqoff/on in bulk allocation

[Jesper Dangaard Brouer <brouer@redhat.com>]

On Fri, 28 Aug 2015 14:44:20 -0500 (CDT)
Christoph Lameter <cl@linux.com> wrote:

> Use the new function that can do allocation while
> interrupts are disabled.  Avoids irq on/off sequences.
> 
> Signed-off-by: Christoph Lameter <cl@linux.com>
> 
> Index: linux/mm/slub.c
> ===================================================================
> --- linux.orig/mm/slub.c	2015-08-28 14:34:59.377234626 -0500
> +++ linux/mm/slub.c	2015-08-28 14:34:59.377234626 -0500
> @@ -2823,30 +2823,23 @@ bool kmem_cache_alloc_bulk(struct kmem_c
>  		void *object = c->freelist;
> 
>  		if (unlikely(!object)) {
> -			local_irq_enable();
>  			/*
>  			 * Invoking slow path likely have side-effect
>  			 * of re-populating per CPU c->freelist
>  			 */
> -			p[i] = __slab_alloc(s, flags, NUMA_NO_NODE,
> +			p[i] = ___slab_alloc(s, flags, NUMA_NO_NODE,
>  					    _RET_IP_, c);
> -			if (unlikely(!p[i])) {
> -				__kmem_cache_free_bulk(s, i, p);
> -				return false;
> -			}
> -			local_irq_disable();
> +			if (unlikely(!p[i]))
> +				goto error;
> +
>  			c = this_cpu_ptr(s->cpu_slab);
>  			continue; /* goto for-loop */
>  		}
> 
>  		/* kmem_cache debug support */
>  		s = slab_pre_alloc_hook(s, flags);
> -		if (unlikely(!s)) {
> -			__kmem_cache_free_bulk(s, i, p);
> -			c->tid = next_tid(c->tid);
> -			local_irq_enable();
> -			return false;
> -		}
> +		if (unlikely(!s))
> +			goto error;
> 
>  		c->freelist = get_freepointer(s, object);
>  		p[i] = object;
> @@ -2866,6 +2859,11 @@ bool kmem_cache_alloc_bulk(struct kmem_c
>  	}
> 
>  	return true;
> +
> +error:
> +	__kmem_cache_free_bulk(s, i, p);

Don't we need to update "tid" here, like:

  c->tid = next_tid(c->tid);

Consider a call to the ordinary kmem_cache_alloc/slab_alloc_node was
in-progress, which get PREEMPT'ed just before it's call to
this_cpu_cmpxchg_double().
 Now, this function gets called and we modify c->freelist, but cannot
get all objects and then fail (goto error).  Although we put-back
objects (via __kmem_cache_free_bulk) don't we want to update c->tid
in-order to make sure the call to this_cpu_cmpxchg_double() retry?

> +	local_irq_enable();
> +	return false;
>  }
>  EXPORT_SYMBOL(kmem_cache_alloc_bulk);


-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>