Re: slub bulk alloc: Extract objects from the per cpu slab

[Andrew Morton <akpm@linux-foundation.org>]

On Wed, 8 Apr 2015 13:13:29 -0500 (CDT) Christoph Lameter <cl@linux.com> wrote:

> First piece: accelleration of retrieval of per cpu objects
> 
> 
> If we are allocating lots of objects then it is advantageous to
> disable interrupts and avoid the this_cpu_cmpxchg() operation to
> get these objects faster. Note that we cannot do the fast operation
> if debugging is enabled.

Why can't we do it if debugging is enabled?

> Note also that the requirement of having
> interrupts disabled avoids having to do processor flag operations.
> 
> Allocate as many objects as possible in the fast way and then fall
> back to the generic implementation for the rest of the objects.

Seems sane.  What's the expected success rate of the initial bulk
allocation attempt?

> --- linux.orig/mm/slub.c
> +++ linux/mm/slub.c
> @@ -2761,7 +2761,32 @@ EXPORT_SYMBOL(kmem_cache_free_bulk);
>  bool kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
>  								void **p)
>  {
> -	return kmem_cache_alloc_bulk(s, flags, size, p);
> +	if (!kmem_cache_debug(s)) {
> +		struct kmem_cache_cpu *c;
> +
> +		/* Drain objects in the per cpu slab */
> +		local_irq_disable();
> +		c = this_cpu_ptr(s->cpu_slab);
> +
> +		while (size) {
> +			void *object = c->freelist;
> +
> +			if (!object)
> +				break;
> +
> +			c->freelist = get_freepointer(s, object);
> +			*p++ = object;
> +			size--;
> +
> +			if (unlikely(flags & __GFP_ZERO))
> +				memset(object, 0, s->object_size);
> +		}
> +		c->tid = next_tid(c->tid);
> +
> +		local_irq_enable();
> +	}
> +
> +	return __kmem_cache_alloc_bulk(s, flags, size, p);

This kmem_cache_cpu.tid logic is a bit opaque.  The low-level
operations seem reasonably well documented but I couldn't find anywhere
which tells me how it all actually works - what is "disambiguation
during cmpxchg" and how do we achieve it?


I'm in two minds about putting
slab-infrastructure-for-bulk-object-allocation-and-freeing-v3.patch and
slub-bulk-alloc-extract-objects-from-the-per-cpu-slab.patch into 4.1. 
They're standalone (ie: no in-kernel callers!) hence harmless, and
merging them will make Jesper's life a bit easier.  But otoh they are
unproven and have no in-kernel callers, so formally they shouldn't be
merged yet.  I suppose we can throw them away again if things don't
work out.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>