From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B872C4167B for ; Mon, 4 Dec 2023 16:58:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5AC16B02DC; Mon, 4 Dec 2023 11:58:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E0AE66B02DD; Mon, 4 Dec 2023 11:58:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD2846B02DE; Mon, 4 Dec 2023 11:58:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BAC006B02DC for ; Mon, 4 Dec 2023 11:58:13 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6EF12802FB for ; Mon, 4 Dec 2023 16:58:13 +0000 (UTC) X-FDA: 81529743666.08.E1F492A Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf03.hostedemail.com (Postfix) with ESMTP id 23A6620026 for ; Mon, 4 Dec 2023 16:58:10 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf03.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701709091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CdoD3EJkYKwdKUzvgkGIFhcvDWIbPAe6anPnyxcLEEQ=; b=yATIzFjRbVK/iCP5H4QbmCuo/nCKowIOIR7hwWi7Xk6Oj1f+LQqBKFdWIWhU/TLAYLrtJA 1XJHb5aJCQCVk08ZAmcNr0hC2h4V76zwabSVTPygRDMJ9gZeoF1Z+VyvK0jShRPdTnwFGz YxhQy1bXonsTp0gK0G4fUAz2169yyzQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf03.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701709091; a=rsa-sha256; cv=none; b=1QdsG262LOw97CXUiAcvOna3w0KZWOvQv5ZXhyT03aLIjzRokDmOjiQNuneot/cxMyXNF5 L5BpYeXy4TBLUe+y5zB0zSh/9ZrONRku0P4IBvkEG7cevWr/mNFFWWaAqs8cFWk+n9V9Rt LbCYx4brynvtdv/9OkJ305udwwbXKaM= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D5CE11F894; Mon, 4 Dec 2023 16:58:08 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BC0D01398A; Mon, 4 Dec 2023 16:58:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WmKLLSAFbmU2eAAAD6G6ig (envelope-from ); Mon, 04 Dec 2023 16:58:08 +0000 Message-ID: <7cd33b23-64f5-a736-ea69-b29e40d42e78@suse.cz> Date: Mon, 4 Dec 2023 17:58:08 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: [PATCH v5 6/9] slub: Delay freezing of partial slabs Content-Language: en-US To: Chengming Zhou , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, roman.gushchin@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chengming Zhou References: <20231102032330.1036151-1-chengming.zhou@linux.dev> <20231102032330.1036151-7-chengming.zhou@linux.dev> <98763097-d05e-40cd-afe0-4df65083d104@linux.dev> From: Vlastimil Babka In-Reply-To: <98763097-d05e-40cd-afe0-4df65083d104@linux.dev> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: +++++ X-Rspamd-Queue-Id: 23A6620026 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 4614kntbphwarkjwoinwxr7f8zob84yb X-HE-Tag: 1701709090-176857 X-HE-Meta: U2FsdGVkX19vAU6Y9zrkNv2HbSAKY+eHHxMHaHnnb5ZOAion2/J0Kgb+tqznqFA4sz75/kbnIL94vhmLanFU92fRxF+aNdUrjeZlkherp2F4CIKVq9i7tMVUTdaIEWHeFn12Sr2+B+KSQJv3TTunLjg3JoLgndNRpERA5bbtydjIADYORADk6LUv+Jc4UWDi99WjctPqWfvU2KSXEoRkBMWvGU0GRX3wBcuDfwMwFvXpwkIqruyZ1Rj+Uyrx0yJi6dNvuE0p7hZ2xghqwlXBSviJmke3fO86ybqUc+uM/fiPYYWGGUkNhtT/kntTcuHptIAShQ4hOg/l50SuWJjejt94NRPhHF47q1VHI1uUHGIZ43SR1C+48jxqGDY9qFWD/49DYLaZi2uevON/aze5vBLqPqX7tKyUvvS6lA5/CKzIOPFlsmkj5GuwIe9hH9B5J3NE9lDAPVda96/2JsgvSn1SRjgRF2NwkKPxVuljQktfyH9VRqbD9aV05InBs7QBVG96uSTdGC3xm8h0zoaQNCO0AuxJbZE4fZzaVDI3vE7GYFnAJWElAft72AY0KTdpI5qrTYkAiNy8vTCtmiOixYGxcO2zOWmAwUnpr9X2HE2B6+VOJRF1hRcx7YlrUu77MfqxjsHSloWF+KRXoIUDhJm7ZQM2k8bs08f6CICoWm0AE8rGo3hsGqfR0rhFg9/O2gxqVNPnByemT6MnmZKWSMssnaGhYz1ZhgXRNqhz5lbq7RFJnfpoTwblnqLvqKBnZupNnK+WWAbx5C9nCb5LU6MFG6QPNJ+KAQePa4tbTPWApXDi15ofXDWEzqpAjEywbnR/G0OC8uYPO4pV4N6BxClKsUOogYZz5kYIOI93b1VsnGXeMgM/vs3QwI/aKap0yvX6C++1eBW0sAHWp7Fmo8Vd2v7hS8AZISkF4toR1B4GBp84lfKr3rhdmOfehT4Dhr4TmvuCGtT+wHy9QLl Iy52A99K qPYuOeMwviYfkDNjB689in/YiAg/nXs8a5E3uQx/z4xXiUQj2tz7vYMjI6M3PahXRl2tvXjDhHUHqiIq3cW9Rd3b74F/JZY7oR7lVSDyK7JmU6hUcImad18he+PqGvyB5482yzh+I2Cf3rEkU3uMuZU3EyFouDLEnjVuVQyq/99ALIhLg3SjCTL6pFykuzk+1m5K2r7ORZAQ+QwGshtg5QnrmIObgqGEh/+6jrsFAEDE8PxxOfokMxPFxxpYBuUK5qo2x1dQRkWca3AY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 12/3/23 11:15, Chengming Zhou wrote: > On 2023/12/3 14:53, Hyeonggon Yoo wrote: >> On Thu, Nov 2, 2023 at 12:25 PM wrote: >>> >>> From: Chengming Zhou >>> >>> Now we will freeze slabs when moving them out of node partial list to >>> cpu partial list, this method needs two cmpxchg_double operations: >>> >>> 1. freeze slab (acquire_slab()) under the node list_lock >>> 2. get_freelist() when pick used in ___slab_alloc() >>> >>> Actually we don't need to freeze when moving slabs out of node partial >>> list, we can delay freezing to when use slab freelist in ___slab_alloc(), >>> so we can save one cmpxchg_double(). >>> >>> And there are other good points: >>> - The moving of slabs between node partial list and cpu partial list >>> becomes simpler, since we don't need to freeze or unfreeze at all. >>> >>> - The node list_lock contention would be less, since we don't need to >>> freeze any slab under the node list_lock. >>> >>> We can achieve this because there is no concurrent path would manipulate >>> the partial slab list except the __slab_free() path, which is now >>> serialized by slab_test_node_partial() under the list_lock. >>> >>> Since the slab returned by get_partial() interfaces is not frozen anymore >>> and no freelist is returned in the partial_context, so we need to use the >>> introduced freeze_slab() to freeze it and get its freelist. >>> >>> Similarly, the slabs on the CPU partial list are not frozen anymore, >>> we need to freeze_slab() on it before use. >>> >>> We can now delete acquire_slab() as it became unused. >>> >>> Signed-off-by: Chengming Zhou >>> Reviewed-by: Vlastimil Babka >>> Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> >>> --- >>> mm/slub.c | 113 +++++++++++------------------------------------------- >>> 1 file changed, 23 insertions(+), 90 deletions(-) >>> >>> diff --git a/mm/slub.c b/mm/slub.c >>> index edf567971679..bcb5b2c4e213 100644 >>> --- a/mm/slub.c >>> +++ b/mm/slub.c >>> @@ -2234,51 +2234,6 @@ static void *alloc_single_from_new_slab(struct kmem_cache *s, >>> return object; >>> } >>> >>> -/* >>> - * Remove slab from the partial list, freeze it and >>> - * return the pointer to the freelist. >>> - * >>> - * Returns a list of objects or NULL if it fails. >>> - */ >>> -static inline void *acquire_slab(struct kmem_cache *s, >>> - struct kmem_cache_node *n, struct slab *slab, >>> - int mode) >> >> Nit: alloc_single_from_partial()'s comment still refers to acquire_slab(). >> > > Ah, right! It should be changed to remove_partial(). > > diff --git a/mm/slub.c b/mm/slub.c > index 437485a2408d..623c17a4cdd6 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -2463,7 +2463,7 @@ static inline void remove_partial(struct kmem_cache_node *n, > } > > /* > - * Called only for kmem_cache_debug() caches instead of acquire_slab(), with a > + * Called only for kmem_cache_debug() caches instead of remove_partial(), with a > * slab from the n->partial list. Remove only a single object from the slab, do > * the alloc_debug_processing() checks and leave the slab on the list, or move > * it to full list if it was the last free object. > > Hi Vlastimil, could you please help to fold it? Done, thanks.