From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ECCFC433FE for ; Tue, 21 Sep 2021 15:37:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 77EB760E08 for ; Tue, 21 Sep 2021 15:37:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 77EB760E08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id BB1E394000E; Tue, 21 Sep 2021 11:37:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B60BA940007; Tue, 21 Sep 2021 11:37:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A500094000E; Tue, 21 Sep 2021 11:37:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 95F13940007 for ; Tue, 21 Sep 2021 11:37:43 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4E3C18249980 for ; Tue, 21 Sep 2021 15:37:43 +0000 (UTC) X-FDA: 78611985606.09.B9EA3CC Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) by imf06.hostedemail.com (Postfix) with ESMTP id D3711801A8B8 for ; Tue, 21 Sep 2021 15:37:42 +0000 (UTC) Received: by mail-io1-f54.google.com with SMTP id s20so18996848ioa.4 for ; Tue, 21 Sep 2021 08:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=bc1nCjtA9E5naFj/iieaeVOdv7rs48Z0WzIV2QbKr+U=; b=OfYiCg1ObQ+EjA8ezpWI/3b5+p47r3DuQ44Hhx2m6hzXqW+bA0IKjb+b7VGHygu21c A1tjp3CHrtC8f9PxzdKgiPgGTfL/xNSPnOmp444ag3J4LxNa8DXLC8ydyUOyk84YdWYw AXbBTwTsQZQSVQRJLk8WwXRdMiMQMeF9y7rq4I2rojpgd3+ciBzpJyptQYb1D/PVs5sV knd62R6tARK2X1cnt6w4RcYMXgYrUY5MItve6qgp4Yfpf3bSTiGc6o5yjPdd4MpPWWkg pYbIE8hWzEeecgYKdrfxcP8D+Rmh4uz0T4pEHvksHWYeTqSfrhv2Xo5Is6Hc1nCzED/j i9aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=bc1nCjtA9E5naFj/iieaeVOdv7rs48Z0WzIV2QbKr+U=; b=DY9WDr/8J3NbqwTG9w0jgVMk5Bb0PvfisJqfEHhrz6STJ0N8vsPRn1eDCZ0QQyrfJU SCw3wOVM8xMVXGEHQSTt9uPy0KrNXSiKmr/G4XCzTGHMW7pdwjn6B9l8NCYVD1gyyIRh woncwWAUQpqndBxrQHIZvpRVU3uUypcwEkn+gyq2gPLg91PPRb9DJZrWfWwX5Q4+xYcw xuBe3YM0Ut1FLUexSVVMDSVK6GAiqViZwjls0xywHnWqEnZTOmld1fnYxTmBDNsQnLyS N+G0wAX6AWZIkKWqCk8aaZRt1/RIKmx2GYli9Z5659Y7MOVme1BqaXh5/bNASUP4mbSs gFDg== X-Gm-Message-State: AOAM531GMn+PnMBOQhAvn4RR2olJ+SyrPIfzG5pzoXGk6yQpP5sUo0k2 RRatVG9KS8T9iUaGxLIQT4pDTw== X-Google-Smtp-Source: ABdhPJxwn39GagTcBMDoxqiNOTdAFFndvbDGREPxOIa25F+Lkj7Wif0shz5JkxgywXozyHTp84Qwnw== X-Received: by 2002:a5d:8458:: with SMTP id w24mr505054ior.168.1632238662030; Tue, 21 Sep 2021 08:37:42 -0700 (PDT) Received: from [192.168.1.30] ([207.135.234.126]) by smtp.gmail.com with ESMTPSA id w4sm636605iou.42.2021.09.21.08.37.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Sep 2021 08:37:41 -0700 (PDT) Subject: Re: [RFC v2 PATCH] mm, sl[au]b: Introduce lockless cache To: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org Cc: Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka , linux-kernel@vger.kernel.org, Matthew Wilcox , John Garry , linux-block@vger.kernel.org, netdev@vger.kernel.org References: <20210920154816.31832-1-42.hyeyoo@gmail.com> From: Jens Axboe Message-ID: Date: Tue, 21 Sep 2021 09:37:40 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20210920154816.31832-1-42.hyeyoo@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D3711801A8B8 X-Stat-Signature: 55gq3ieosg5im6cs7phjiyj47ngcj1r4 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=OfYiCg1O; spf=pass (imf06.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.54 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none X-HE-Tag: 1632238662-992383 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > @@ -424,6 +431,57 @@ kmem_cache_create(const char *name, unsigned int size, unsigned int align, > } > EXPORT_SYMBOL(kmem_cache_create); > > +/** > + * kmem_cache_alloc_cached - try to allocate from cache without lock > + * @s: slab cache > + * @flags: SLAB flags > + * > + * Try to allocate from cache without lock. If fails, fill the lockless cache > + * using bulk alloc API > + * > + * Be sure that there's no race condition. > + * Must create slab cache with SLAB_LOCKLESS_CACHE flag to use this function. > + * > + * Return: a pointer to free object on allocation success, NULL on failure. > + */ > +void *kmem_cache_alloc_cached(struct kmem_cache *s, gfp_t gfpflags) > +{ > + struct kmem_lockless_cache *cache = this_cpu_ptr(s->cache); > + > + BUG_ON(!(s->flags & SLAB_LOCKLESS_CACHE)); > + > + if (cache->size) /* fastpath without lock */ > + return cache->queue[--cache->size]; > + > + /* slowpath */ > + cache->size = kmem_cache_alloc_bulk(s, gfpflags, > + KMEM_LOCKLESS_CACHE_QUEUE_SIZE, cache->queue); > + if (cache->size) > + return cache->queue[--cache->size]; > + else > + return NULL; > +} > +EXPORT_SYMBOL(kmem_cache_alloc_cached); How does this work for preempt? You seem to assume that the function is invoked with preempt disabled, but then it could only be used with GFP_ATOMIC. There are basically two types of use cases for this: 1) Freeing can happen from interrupts 2) Freeing cannot happen from interrupts What I implemented for IOPOLL doesn't need to care about interrupts, hence preemption disable is enough. But we do need that, at least. And if you don't care about users that free from irq/softirq, then that should be mentioned. Locking context should be mentioned, too. The above may be just fine IFF both alloc and free are protected by a lock higher up. If not, both need preemption disabled and GFP_ATOMIC. I'd suggest making the get/put cpu part of the API internally. > +/** > + * kmem_cache_free_cached - return object to cache > + * @s: slab cache > + * @p: pointer to free > + */ > +void kmem_cache_free_cached(struct kmem_cache *s, void *p) > +{ > + struct kmem_lockless_cache *cache = this_cpu_ptr(s->cache); > + > + BUG_ON(!(s->flags & SLAB_LOCKLESS_CACHE)); Don't use BUG_ON, just do: if (WARN_ON_ONCE(!(s->flags & SLAB_LOCKLESS_CACHE))) { kmem_cache_free(s, p); return; } -- Jens Axboe