Re: [PATCHv2] zsmalloc: allow only one active pool compaction context

[Sergey Senozhatsky <senozhatsky@chromium.org>]

On (23/04/17 17:41), Andrew Morton wrote:
> On Mon, 17 Apr 2023 22:54:20 +0900 Sergey Senozhatsky <senozhatsky@chromium.org> wrote:
> > Introduce pool compaction mutex and permit only one compaction
> > context at a time. This reduces overall pool->lock contention.
> 
> That isn't what the patch does!  Perhaps an earlier version used a mutex?

Oh, yes.

[..]
> > @@ -2274,6 +2275,9 @@ unsigned long zs_compact(struct zs_pool *pool)
> >  	struct size_class *class;
> >  	unsigned long pages_freed = 0;
> >  
> > +	if (atomic_xchg(&pool->compaction_in_progress, 1))
> > +		return 0;
> > +
> 
> A code comment might be appropriate here.

OK.

> Is the spin_is_contended() test in __zs_compact() still relevant?

I'd say yes, pool->lock is "big pool lock", we use it for everything:
zs_map_object() when we read objects, zs_malloc() when we allocate objects,
zs_free() when we free objects, etc.

> And....  single-threading the operation seems a pretty sad way of
> addressing a contention issue.  zs_compact() is fairly computationally
> expensive - surely a large machine would like to be able to
> concurrently run many instances of zs_compact()?

Compaction is effective only when zspool suffers from internal fragmentation.
Concurrent compaction threads iterate size classes in the same order and are
likely to compete for pool->lock to just find out that previous pool->lock
owner has compacted the class already and there isn't much left to do.

As of concurrent compaction on big machines, the thing is - concurrent
compaction doesn't really happen. We always have just one thread compacting
size classes under pool->lock, the remaining compaction threads just spin on
pool->lock. I believe it used to be slightly different in the past when we
had per size-class lock instead of "global" pool->lock. Commit c0547d0b6a4b
("zsmalloc: consolidate zs_pool's migrate_lock and size_class's locks") has
basically made compaction single-threaded.