From mboxrd@z Thu Jan 1 00:00:00 1970 From: Qi Zheng Date: Sat, 29 Jul 2023 16:47:57 +0800 Subject: [Cluster-devel] [PATCH v3 05/49] mm: shrinker: add infrastructure for dynamically allocating shrinker In-Reply-To: References: <20230727080502.77895-1-zhengqi.arch@bytedance.com> <20230727080502.77895-6-zhengqi.arch@bytedance.com> Message-ID: <5e50711c-a616-f95f-d6d2-c69627ac3cf0@bytedance.com> List-Id: To: cluster-devel.redhat.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi Simon, On 2023/7/28 20:17, Simon Horman wrote: > On Thu, Jul 27, 2023 at 04:04:18PM +0800, Qi Zheng wrote: >> Currently, the shrinker instances can be divided into the following three >> types: >> >> a) global shrinker instance statically defined in the kernel, such as >> workingset_shadow_shrinker. >> >> b) global shrinker instance statically defined in the kernel modules, such >> as mmu_shrinker in x86. >> >> c) shrinker instance embedded in other structures. >> >> For case a, the memory of shrinker instance is never freed. For case b, >> the memory of shrinker instance will be freed after synchronize_rcu() when >> the module is unloaded. For case c, the memory of shrinker instance will >> be freed along with the structure it is embedded in. >> >> In preparation for implementing lockless slab shrink, we need to >> dynamically allocate those shrinker instances in case c, then the memory >> can be dynamically freed alone by calling kfree_rcu(). >> >> So this commit adds the following new APIs for dynamically allocating >> shrinker, and add a private_data field to struct shrinker to record and >> get the original embedded structure. >> >> 1. shrinker_alloc() >> >> Used to allocate shrinker instance itself and related memory, it will >> return a pointer to the shrinker instance on success and NULL on failure. >> >> 2. shrinker_register() >> >> Used to register the shrinker instance, which is same as the current >> register_shrinker_prepared(). >> >> 3. shrinker_free() >> >> Used to unregister (if needed) and free the shrinker instance. >> >> In order to simplify shrinker-related APIs and make shrinker more >> independent of other kernel mechanisms, subsequent submissions will use >> the above API to convert all shrinkers (including case a and b) to >> dynamically allocated, and then remove all existing APIs. >> >> This will also have another advantage mentioned by Dave Chinner: >> >> ``` >> The other advantage of this is that it will break all the existing >> out of tree code and third party modules using the old API and will >> no longer work with a kernel using lockless slab shrinkers. They >> need to break (both at the source and binary levels) to stop bad >> things from happening due to using uncoverted shrinkers in the new > > nit: uncoverted -> unconverted Thanks. Will fix. > >> setup. >> ``` >> >> Signed-off-by: Qi Zheng > > ... > >> diff --git a/mm/shrinker_debug.c b/mm/shrinker_debug.c >> index f1becfd45853..506257585408 100644 >> --- a/mm/shrinker_debug.c >> +++ b/mm/shrinker_debug.c >> @@ -191,6 +191,20 @@ int shrinker_debugfs_add(struct shrinker *shrinker) >> return 0; >> } >> >> +int shrinker_debugfs_name_alloc(struct shrinker *shrinker, const char *fmt, >> + va_list ap) >> +{ >> + shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); >> + >> + return shrinker->name ? 0 : -ENOMEM; >> +} >> + >> +void shrinker_debugfs_name_free(struct shrinker *shrinker) >> +{ >> + kfree_const(shrinker->name); >> + shrinker->name = NULL; >> +} >> + > > These functions have no prototype in this file, > perhaps internal.h should be included? The compiler can find these implementations, so I don't think there is a need to include internal.h here? Thanks, Qi > >> int shrinker_debugfs_rename(struct shrinker *shrinker, const char *fmt, ...) >> { >> struct dentry *entry; > > ... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5455AC001DC for ; Sat, 29 Jul 2023 08:48:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690620507; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=AbdgPGpGzwv3Ba9ss/qbC8w4dcIJ6EfJ8tT9+ztVBsQ=; b=SREZKzl7LU86IXjT4RqENBsiDfEVdY9bh4gFg9CTW44meV4tkbqP+R0z+3ctG3tXI/iDOq DPYjBhQaEYAVMYXYTphx+UU0X6WPP8yB0Bvq3Li3b8yzF4oFCwQCHo11emusiuxFYRTgkB iv/Fw1nJq0tFp/jmzTCpR64XLrfmyqI= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-pkzil9_vO5u-5tFpIqdZPA-1; Sat, 29 Jul 2023 04:48:20 -0400 X-MC-Unique: pkzil9_vO5u-5tFpIqdZPA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A3F623C0F180; Sat, 29 Jul 2023 08:48:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6ED7C57964; Sat, 29 Jul 2023 08:48:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 784FD1946A6A; Sat, 29 Jul 2023 08:48:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E643B194658C for ; Sat, 29 Jul 2023 08:48:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D3316207B2DF; Sat, 29 Jul 2023 08:48:15 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CB1AB2017D0A for ; Sat, 29 Jul 2023 08:48:15 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-inbound-delivery-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AF4B080027F for ; Sat, 29 Jul 2023 08:48:15 +0000 (UTC) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-304-VdMso74ZPK-3feFth3Sq4w-1; Sat, 29 Jul 2023 04:48:09 -0400 X-MC-Unique: VdMso74ZPK-3feFth3Sq4w-1 Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-682b1768a0bso692001b3a.0 for ; Sat, 29 Jul 2023 01:48:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690620488; x=1691225288; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AbdgPGpGzwv3Ba9ss/qbC8w4dcIJ6EfJ8tT9+ztVBsQ=; b=K/399t/upiu5ohbHQzh2dpXCXGo8sihddHWXsWxmBQutGZAvCOmuXhtaX/eyz9b7oQ iosxFpx2BCDDYSdowQccu7FBzyITs+MEkc4uRK7gju1yuwGOZbbbl6Gyq8TOHhzkkeX/ 5GcdcvihrWbGswXfEOdXMFK1jrQgnkMtFfy6dSTO+gaf7XIg18jaIP+jGe+G3dk7YGL6 AdNQuks4mgyqymrR0yfPzkUqFXys0eHOHyIw0hxmgNgC0pJudeVgZJ0madhCGIRM7xer rvhUxJKI7LaszGBdv1cokObODzjkdV2kES0MYx+7w6Z3zEOdVZW9pfkPTf/sITy/H2TB MR2Q== X-Gm-Message-State: ABy/qLabn2rFdCG2I2ywdXuxOEnubYFE/9JvRqoEC+N+AamD0gWZkn1R wuQpxIvcxfwSUhit+PVwddVTsw== X-Google-Smtp-Source: APBJJlGU4WvDtIE3roPXfr2RIP3jnPqKjXBKgd99me7vK4BYh3P3V9BBkMbm8JQkpcQK7ib8McexOg== X-Received: by 2002:a05:6a00:32c8:b0:67f:7403:1fe8 with SMTP id cl8-20020a056a0032c800b0067f74031fe8mr1763906pfb.3.1690620488142; Sat, 29 Jul 2023 01:48:08 -0700 (PDT) Received: from ?IPV6:fdbd:ff1:ce00:1c25:884:3ed:e1db:b610? ([240e:694:e21:b::2]) by smtp.gmail.com with ESMTPSA id s1-20020a62e701000000b00687087d8bc3sm2935245pfh.141.2023.07.29.01.47.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 29 Jul 2023 01:48:07 -0700 (PDT) Message-ID: <5e50711c-a616-f95f-d6d2-c69627ac3cf0@bytedance.com> Date: Sat, 29 Jul 2023 16:47:57 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 To: Simon Horman References: <20230727080502.77895-1-zhengqi.arch@bytedance.com> <20230727080502.77895-6-zhengqi.arch@bytedance.com> From: Qi Zheng In-Reply-To: X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Subject: Re: [Cluster-devel] [PATCH v3 05/49] mm: shrinker: add infrastructure for dynamically allocating shrinker X-BeenThere: cluster-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: "\[Cluster devel\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, djwong@kernel.org, roman.gushchin@linux.dev, david@fromorbit.com, dri-devel@lists.freedesktop.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, dm-devel@redhat.com, linux-mtd@lists.infradead.org, cel@kernel.org, x86@kernel.org, steven.price@arm.com, cluster-devel@redhat.com, xen-devel@lists.xenproject.org, linux-ext4@vger.kernel.org, paulmck@kernel.org, linux-arm-msm@vger.kernel.org, linux-nfs@vger.kernel.org, rcu@vger.kernel.org, linux-bcache@vger.kernel.org, yujie.liu@intel.com, vbabka@suse.cz, linux-raid@vger.kernel.org, brauner@kernel.org, tytso@mit.edu, gregkh@linuxfoundation.org, muchun.song@linux.dev, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-xfs@vger.kernel.org, senozhatsky@chromium.org, netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org, linux-erofs@lists.ozlabs.org, linux-btrfs@vger.kernel.org, tkhai@ya.ru Errors-To: cluster-devel-bounces@redhat.com Sender: "Cluster-devel" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: bytedance.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID: <20230729084757.K4lWOI5WI0a7fxfB1Hcd33WR4XQm20LRXhecVed84UM@z> Hi Simon, On 2023/7/28 20:17, Simon Horman wrote: > On Thu, Jul 27, 2023 at 04:04:18PM +0800, Qi Zheng wrote: >> Currently, the shrinker instances can be divided into the following three >> types: >> >> a) global shrinker instance statically defined in the kernel, such as >> workingset_shadow_shrinker. >> >> b) global shrinker instance statically defined in the kernel modules, such >> as mmu_shrinker in x86. >> >> c) shrinker instance embedded in other structures. >> >> For case a, the memory of shrinker instance is never freed. For case b, >> the memory of shrinker instance will be freed after synchronize_rcu() when >> the module is unloaded. For case c, the memory of shrinker instance will >> be freed along with the structure it is embedded in. >> >> In preparation for implementing lockless slab shrink, we need to >> dynamically allocate those shrinker instances in case c, then the memory >> can be dynamically freed alone by calling kfree_rcu(). >> >> So this commit adds the following new APIs for dynamically allocating >> shrinker, and add a private_data field to struct shrinker to record and >> get the original embedded structure. >> >> 1. shrinker_alloc() >> >> Used to allocate shrinker instance itself and related memory, it will >> return a pointer to the shrinker instance on success and NULL on failure. >> >> 2. shrinker_register() >> >> Used to register the shrinker instance, which is same as the current >> register_shrinker_prepared(). >> >> 3. shrinker_free() >> >> Used to unregister (if needed) and free the shrinker instance. >> >> In order to simplify shrinker-related APIs and make shrinker more >> independent of other kernel mechanisms, subsequent submissions will use >> the above API to convert all shrinkers (including case a and b) to >> dynamically allocated, and then remove all existing APIs. >> >> This will also have another advantage mentioned by Dave Chinner: >> >> ``` >> The other advantage of this is that it will break all the existing >> out of tree code and third party modules using the old API and will >> no longer work with a kernel using lockless slab shrinkers. They >> need to break (both at the source and binary levels) to stop bad >> things from happening due to using uncoverted shrinkers in the new > > nit: uncoverted -> unconverted Thanks. Will fix. > >> setup. >> ``` >> >> Signed-off-by: Qi Zheng > > ... > >> diff --git a/mm/shrinker_debug.c b/mm/shrinker_debug.c >> index f1becfd45853..506257585408 100644 >> --- a/mm/shrinker_debug.c >> +++ b/mm/shrinker_debug.c >> @@ -191,6 +191,20 @@ int shrinker_debugfs_add(struct shrinker *shrinker) >> return 0; >> } >> >> +int shrinker_debugfs_name_alloc(struct shrinker *shrinker, const char *fmt, >> + va_list ap) >> +{ >> + shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); >> + >> + return shrinker->name ? 0 : -ENOMEM; >> +} >> + >> +void shrinker_debugfs_name_free(struct shrinker *shrinker) >> +{ >> + kfree_const(shrinker->name); >> + shrinker->name = NULL; >> +} >> + > > These functions have no prototype in this file, > perhaps internal.h should be included? The compiler can find these implementations, so I don't think there is a need to include internal.h here? Thanks, Qi > >> int shrinker_debugfs_rename(struct shrinker *shrinker, const char *fmt, ...) >> { >> struct dentry *entry; > > ...