linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Hyeonggon Yoo <42.hyeyoo@gmail.com>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Christoph Lameter <cl@linux.com>,
	David Rientjes <rientjes@google.com>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Pekka Enberg <penberg@kernel.org>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Matthew Wilcox <willy@infradead.org>,
	patches@lists.linux.dev, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 11/12] mm, slub: don't aggressively inline with CONFIG_SLUB_TINY
Date: Mon, 28 Nov 2022 22:19:19 +0900	[thread overview]
Message-ID: <Y4S1V9MNglgserVG@hyeyoo> (raw)
In-Reply-To: <20221121171202.22080-12-vbabka@suse.cz>

On Mon, Nov 21, 2022 at 06:12:01PM +0100, Vlastimil Babka wrote:
> SLUB fastpaths use __always_inline to avoid function calls. With
> CONFIG_SLUB_TINY we would rather save the memory. Add a
> __fastpath_inline macro that's __always_inline normally but empty with
> CONFIG_SLUB_TINY.
> 
> bloat-o-meter results on x86_64 mm/slub.o:
> 
> add/remove: 3/1 grow/shrink: 1/8 up/down: 865/-1784 (-919)
> Function                                     old     new   delta
> kmem_cache_free                               20     281    +261
> slab_alloc_node.isra                           -     245    +245
> slab_free.constprop.isra                       -     231    +231
> __kmem_cache_alloc_lru.isra                    -     128    +128
> __kmem_cache_release                          88      83      -5
> __kmem_cache_create                         1446    1436     -10
> __kmem_cache_free                            271     142    -129
> kmem_cache_alloc_node                        330     127    -203
> kmem_cache_free_bulk.part                    826     613    -213
> __kmem_cache_alloc_node                      230      10    -220
> kmem_cache_alloc_lru                         325      12    -313
> kmem_cache_alloc                             325      10    -315
> kmem_cache_free.part                         376       -    -376
> Total: Before=26103, After=25184, chg -3.52%
> 
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
> ---
>  mm/slub.c | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 7f1cd702c3b4..d54466e76503 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -187,6 +187,12 @@ do {					\
>  #define USE_LOCKLESS_FAST_PATH()	(false)
>  #endif
>  
> +#ifndef CONFIG_SLUB_TINY
> +#define __fastpath_inline __always_inline
> +#else
> +#define __fastpath_inline
> +#endif
> +
>  #ifdef CONFIG_SLUB_DEBUG
>  #ifdef CONFIG_SLUB_DEBUG_ON
>  DEFINE_STATIC_KEY_TRUE(slub_debug_enabled);
> @@ -3386,7 +3392,7 @@ static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
>   *
>   * Otherwise we can simply pick the next object from the lockless free list.
>   */
> -static __always_inline void *slab_alloc_node(struct kmem_cache *s, struct list_lru *lru,
> +static __fastpath_inline void *slab_alloc_node(struct kmem_cache *s, struct list_lru *lru,
>  		gfp_t gfpflags, int node, unsigned long addr, size_t orig_size)
>  {
>  	void *object;
> @@ -3412,13 +3418,13 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s, struct list_l
>  	return object;
>  }
>  
> -static __always_inline void *slab_alloc(struct kmem_cache *s, struct list_lru *lru,
> +static __fastpath_inline void *slab_alloc(struct kmem_cache *s, struct list_lru *lru,
>  		gfp_t gfpflags, unsigned long addr, size_t orig_size)
>  {
>  	return slab_alloc_node(s, lru, gfpflags, NUMA_NO_NODE, addr, orig_size);
>  }
>  
> -static __always_inline
> +static __fastpath_inline
>  void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
>  			     gfp_t gfpflags)
>  {
> @@ -3733,7 +3739,7 @@ static void do_slab_free(struct kmem_cache *s,
>  }
>  #endif /* CONFIG_SLUB_TINY */
>  
> -static __always_inline void slab_free(struct kmem_cache *s, struct slab *slab,
> +static __fastpath_inline void slab_free(struct kmem_cache *s, struct slab *slab,
>  				      void *head, void *tail, void **p, int cnt,
>  				      unsigned long addr)
>  {
> -- 
> 2.38.1

Acked-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>

-- 
Thanks,
Hyeonggon


  reply	other threads:[~2022-11-28 13:19 UTC|newest]

Thread overview: 60+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-21 17:11 [PATCH 00/12] Introduce CONFIG_SLUB_TINY and deprecate SLOB Vlastimil Babka
2022-11-21 17:11 ` [PATCH 01/12] mm, slab: ignore hardened usercopy parameters when disabled Vlastimil Babka
2022-11-21 21:35   ` Kees Cook
2022-11-23 14:23     ` Vlastimil Babka
2022-11-24 11:16       ` Hyeonggon Yoo
2022-11-24 11:26         ` Vlastimil Babka
2022-11-24 12:33       ` Hyeonggon Yoo
2022-11-21 17:11 ` [PATCH 02/12] mm, slub: add CONFIG_SLUB_TINY Vlastimil Babka
2022-11-24  1:08   ` Roman Gushchin
2022-11-24 11:33   ` Hyeonggon Yoo
2022-11-25  7:55     ` Vlastimil Babka
2022-11-21 17:11 ` [PATCH 03/12] mm, slub: disable SYSFS support with CONFIG_SLUB_TINY Vlastimil Babka
2022-11-24  1:12   ` Roman Gushchin
2022-11-24  9:00     ` Vlastimil Babka
2022-11-21 17:11 ` [PATCH 04/12] mm, slub: retain no free slabs on partial list " Vlastimil Babka
2022-11-24  1:12   ` Roman Gushchin
2022-11-24 11:38   ` Hyeonggon Yoo
2022-11-21 17:11 ` [PATCH 05/12] mm, slub: lower the default slub_max_order " Vlastimil Babka
2022-11-24  1:16   ` Roman Gushchin
2022-11-24 11:40   ` Hyeonggon Yoo
2022-11-21 17:11 ` [PATCH 06/12] mm, slub: don't create kmalloc-rcl caches " Vlastimil Babka
2022-11-23 13:53   ` Vlastimil Babka
2022-11-24 12:06     ` Hyeonggon Yoo
2022-11-24 12:12       ` Vlastimil Babka
2022-11-24 12:55         ` Hyeonggon Yoo
2022-11-24 13:23     ` Hyeonggon Yoo
2022-11-24 14:25       ` Hyeonggon Yoo
2022-11-21 17:11 ` [PATCH 07/12] mm, slab: ignore SLAB_RECLAIM_ACCOUNT " Vlastimil Babka
2022-11-24  1:20   ` Roman Gushchin
2022-11-24  9:09     ` Vlastimil Babka
2022-11-24  9:21       ` Christoph Lameter
2022-11-27 23:11   ` Vlastimil Babka
2022-11-21 17:11 ` [PATCH 08/12] mm, slub: refactor free debug processing Vlastimil Babka
2022-11-27 10:18   ` Hyeonggon Yoo
2022-11-21 17:11 ` [PATCH 09/12] mm, slub: split out allocations from pre/post hooks Vlastimil Babka
2022-11-27 10:54   ` Hyeonggon Yoo
2022-11-27 23:01     ` Vlastimil Babka
2022-11-28 13:06       ` Hyeonggon Yoo
2022-11-21 17:12 ` [PATCH 10/12] mm, slub: remove percpu slabs with CONFIG_SLUB_TINY Vlastimil Babka
2022-11-27 11:05   ` Hyeonggon Yoo
2022-12-12 10:54     ` Vlastimil Babka
2022-12-12 13:11       ` Dennis Zhou
2022-12-13  3:04         ` Baoquan He
2022-12-13 14:02           ` Hyeonggon Yoo
2022-12-18 10:16   ` Hyeonggon Yoo
2022-11-21 17:12 ` [PATCH 11/12] mm, slub: don't aggressively inline " Vlastimil Babka
2022-11-28 13:19   ` Hyeonggon Yoo [this message]
2022-11-21 17:12 ` [PATCH 12/12] mm, slob: rename CONFIG_SLOB to CONFIG_SLOB_DEPRECATED Vlastimil Babka
2022-11-21 18:41   ` Aaro Koskinen
2022-11-21 19:42   ` Vlastimil Babka
2022-11-22  6:47   ` Damien Le Moal
2022-11-22 16:08   ` Arnd Bergmann
2022-11-24  1:21   ` Roman Gushchin
2022-12-02 17:59   ` Palmer Dabbelt
2022-12-05 12:25     ` Damien Le Moal
2022-12-13 13:41   ` Hyeonggon Yoo
2022-11-22 16:33 ` [PATCH 00/12] Introduce CONFIG_SLUB_TINY and deprecate SLOB Arnd Bergmann
2022-11-22 16:59   ` Vlastimil Babka
2022-11-22 17:15     ` Arnd Bergmann
2022-11-24 20:30 ` Mike Rapoport

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Y4S1V9MNglgserVG@hyeyoo \
    --to=42.hyeyoo@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=cl@linux.com \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=patches@lists.linux.dev \
    --cc=penberg@kernel.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=torvalds@linux-foundation.org \
    --cc=vbabka@suse.cz \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).