Re: [PATCH v2 1/1] mm/vmalloc: remove preempt_disable/enable when do preloading

[Uladzislau Rezki <urezki@gmail.com>]

On Mon, Oct 14, 2019 at 03:13:08PM +0200, Michal Hocko wrote:
> On Fri 11-10-19 00:33:18, Uladzislau Rezki (Sony) wrote:
> > Get rid of preempt_disable() and preempt_enable() when the
> > preload is done for splitting purpose. The reason is that
> > calling spin_lock() with disabled preemtion is forbidden in
> > CONFIG_PREEMPT_RT kernel.
> 
> I think it would be really helpful to describe why the preemption was
> disabled in that path. Some of that is explained in the comment but the
> changelog should mention that explicitly.
>  
Will do that, makes sense.

> > Therefore, we do not guarantee that a CPU is preloaded, instead
> > we minimize the case when it is not with this change.
> > 
> > For example i run the special test case that follows the preload
> > pattern and path. 20 "unbind" threads run it and each does
> > 1000000 allocations. Only 3.5 times among 1000000 a CPU was
> > not preloaded. So it can happen but the number is negligible.
> > 
> > V1 -> V2:
> >   - move __this_cpu_cmpxchg check when spin_lock is taken,
> >     as proposed by Andrew Morton
> >   - add more explanation in regard of preloading
> >   - adjust and move some comments
> > 
> > Fixes: 82dd23e84be3 ("mm/vmalloc.c: preload a CPU with one object for split purpose")
> > Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
> > ---
> >  mm/vmalloc.c | 50 +++++++++++++++++++++++++++++++++-----------------
> >  1 file changed, 33 insertions(+), 17 deletions(-)
> > 
> > diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> > index e92ff5f7dd8b..f48cd0711478 100644
> > --- a/mm/vmalloc.c
> > +++ b/mm/vmalloc.c
> > @@ -969,6 +969,19 @@ adjust_va_to_fit_type(struct vmap_area *va,
> >  			 * There are a few exceptions though, as an example it is
> >  			 * a first allocation (early boot up) when we have "one"
> >  			 * big free space that has to be split.
> > +			 *
> > +			 * Also we can hit this path in case of regular "vmap"
> > +			 * allocations, if "this" current CPU was not preloaded.
> > +			 * See the comment in alloc_vmap_area() why. If so, then
> > +			 * GFP_NOWAIT is used instead to get an extra object for
> > +			 * split purpose. That is rare and most time does not
> > +			 * occur.
> > +			 *
> > +			 * What happens if an allocation gets failed. Basically,
> > +			 * an "overflow" path is triggered to purge lazily freed
> > +			 * areas to free some memory, then, the "retry" path is
> > +			 * triggered to repeat one more time. See more details
> > +			 * in alloc_vmap_area() function.
> >  			 */
> >  			lva = kmem_cache_alloc(vmap_area_cachep, GFP_NOWAIT);
> 
> This doesn't seem to have anything to do with the patch. Have you
> considered to make it a patch on its own? Btw. I find this comment very
> useful!
> 
Makes sense, will make it as separate patch.

> >  			if (!lva)
> > @@ -1078,31 +1091,34 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
> >  
> >  retry:
> >  	/*
> > -	 * Preload this CPU with one extra vmap_area object to ensure
> > -	 * that we have it available when fit type of free area is
> > -	 * NE_FIT_TYPE.
> > +	 * Preload this CPU with one extra vmap_area object. It is used
> > +	 * when fit type of free area is NE_FIT_TYPE. Please note, it
> > +	 * does not guarantee that an allocation occurs on a CPU that
> > +	 * is preloaded, instead we minimize the case when it is not.
> > +	 * It can happen because of migration, because there is a race
> > +	 * until the below spinlock is taken.
> 
> s@migration@cpu migration@ because migration without on its own is quite
> ambiguous, especially in the MM code where it usually refers to memory.
> 
Thanks, will update it.


Thank you for the comments!

--
Vlad Rezki