From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9DC3EB64DC for ; Mon, 10 Jul 2023 22:17:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 560568D0005; Mon, 10 Jul 2023 18:17:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E8928D0001; Mon, 10 Jul 2023 18:17:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 313D28D0005; Mon, 10 Jul 2023 18:17:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 220A48D0001 for ; Mon, 10 Jul 2023 18:17:11 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8BF8DC0299 for ; Mon, 10 Jul 2023 22:17:10 +0000 (UTC) X-FDA: 80997113820.03.512F412 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf22.hostedemail.com (Postfix) with ESMTP id 1F02CC0011 for ; Mon, 10 Jul 2023 22:17:07 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=TMlHIwao; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf22.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689027428; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8sEk/KJTsk1SrESg8tiLSDNDP7ej4yGxTZP0FlmemOg=; b=dc+EjOKTsjewh+rgCpmxRPUqcpkqI/GXFpCSk4NFItYtrDHT2MNRT2NeHv3q81DUKcJjWh 375Sc4T+mq0HowAJXeY5LEFGlPIviycgwLA+pde/6hTnnIuqg2iMlKfPRz10FpeunEJucq NPvAm8c3Kg69q7jtEXOC7/1b9FBKIK0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=TMlHIwao; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf22.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689027428; a=rsa-sha256; cv=none; b=emgzQB0K3wy7zyY4eQYm6yuJBEFsiiYLJWqeklrEAjd6XM9Nswd2i3YqHJLHLZ1N75eF9B MD2MgaV3YfFQPkk2OogtT8//yVehvYniiI8C7zX8TgSFqtS322bItr8/QsXEDPALRe50FX vGwHYiBSJ5igT98jL1TOhqsn/QNrlf8= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1b8ad356f03so32352665ad.1 for ; Mon, 10 Jul 2023 15:17:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689027427; x=1691619427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=8sEk/KJTsk1SrESg8tiLSDNDP7ej4yGxTZP0FlmemOg=; b=TMlHIwaoW2MqvvQrb+GxBUD2jQf87Dxwhnffklm6HxTH+58pH/x5MITcX7ZszAJkJJ 4TktvHWuKZcPfXUj8r8OwkPLXzxUXKycOd2esMiVNcVuRBEECjuSdCNG+60BBvIJq1G2 fVwDLn3CXLYRJN81aJnZxB5Rxau7VT3Qi67a8Mfzp7Jb2Pp9R5EYfkeqnWZl8z+4hdZO 9/jrpCP92YDEh41Sko9USDY20FElxmGZi9PRjDOxm/MRy4qI272MjJCy9v2BWSAMrPJ8 gG9xDHCVyY6xzGncFjqiLaFGgwaPkoLZEIH88c0f1tYjQFAbrVN8ILJApBb2+v77/v8Y lciA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689027427; x=1691619427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=8sEk/KJTsk1SrESg8tiLSDNDP7ej4yGxTZP0FlmemOg=; b=a5ScwS3vrR0++Qrfs390o/m9/kg5ppSNZn/ZvidpqlZi+P4rCbuPX4ar6rmNtp7UHE bFk8MhhBErj5lyrWTD5g2csQgs9705CMbFgB13TIil/b+QIaYDV3YSDb9UGlRkLk7iyv xE2tiw5GoOS/DnLrVGkaVGHQ+NLNdQN84TSWS+6JWkc7QebJ8FUpE46JT55DltkM5cYb Uj3kpMOsdRVS1VBg6SzOZX5jl8gL/jWX1YwzZ4xKNSBUDkdGYNZuJLcqX9roo95VqZMy jd5grbagb/Bm0lASoJC7iZBRmXivGxNh4OUgcD4nkd0TSdKgyDIYt5ALRrCzhrNYAuhB 9qDw== X-Gm-Message-State: ABy/qLbuozTpzPQjh7/RZP93RLtxpXXSvEMdTUbpWv8TBB/I+2nUrOkN 0jpQj0XX1zpizNn3qms4we4= X-Google-Smtp-Source: APBJJlE2PFoBOjwwWXH692IGmXgN5v+4mpYQ8RniZs75yUZ+ETbiOcAqYNP4ZOECpnggUuniEUz51A== X-Received: by 2002:a17:902:e852:b0:1b9:ea60:cd8a with SMTP id t18-20020a170902e85200b001b9ea60cd8amr1588226plg.50.1689027427391; Mon, 10 Jul 2023 15:17:07 -0700 (PDT) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:201:1922:28f0:ba3b:368e]) by smtp.gmail.com with ESMTPSA id c23-20020a170902cb1700b001b9bebbc621sm376425ply.136.2023.07.10.15.17.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 15:17:06 -0700 (PDT) From: Minchan Kim To: Andrew Morton Cc: linux-mm , Jens Axboe , Konrad Rzeszutek Wilk , Seth Jennings , Sergey Senozhatsky , Minchan Kim Subject: [PATCH 2/3] zram: support frontswap Date: Mon, 10 Jul 2023 15:16:58 -0700 Message-ID: <20230710221659.2473460-3-minchan@kernel.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230710221659.2473460-1-minchan@kernel.org> References: <20230710221659.2473460-1-minchan@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 1F02CC0011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: dxz4goxwebhfq51w73m4utd8q48j3fch X-HE-Tag: 1689027427-153961 X-HE-Meta: U2FsdGVkX1+BrvIJepSHJpsTdjUASRUCPs7OTRev1FfS6E48xAOURzImsU+tjErFFIrxYZ8MMNP7E/v4v7jtnzcUHsvTKi9l+iCLxBlMx8Ag+nNDzo3OJ2uAevv3W2focQj7CnajuGR5VPoVRQ5FtybCxjhigffsF5TZdUWiixrzRAaqGBIHmtiSJaIJ574Myo4u75GAqhxF727oH+uD68EkfcEL7UY3Z1eRBjE4XgHr9XzAaE4OBu8nDBEeHUbfHrjICDX1hPRhc9P9tBz69YvMa1sZpZrsM+0lWJhWxh35EnONvMkPHtn49zuXsD5Ox3oRqq22HF5lsmGohumwRSYpY4ng2cV8Z80BKEPNnNVg314cn8KQ4mqQ1MwLgnME2wq+rRpRTVgA7NQ0CPRzHKVenbjOcwd4wZfxj/Ue884zT9VXOSO0IsBYbQQDx68YuNAqmlG59OaCpmZOlalGaEZqQF++51AE0wkgQwbBQrTDK8lll3FUMs+Ts8plNVG1N4O/kvsK++ya/n/EGUyCvMTxIobEBI+JH2SRzmHF19MkQc3uKrfdrzoQOumt0jHomQAlsgQeQR/gHGDtwWxrrhVdKBTB0gATcTW0WY3q28XAmDa/ZjqSocKTH5NkMuObcSyDKoktVjoBGU817TNEsoXG9sElGu0/P8jGYCbbfGnkoogz92vY60SPdLL1RcQ6s7Hd4dFDlNSMrHamB0J+0Y27jQtABF8difGKpHBfkoUyZZO4NLyxuu2+/hAJ3vmLcfPAuUcc0fJzpWyqCptoilsVwOon4cVHJlf4ng7js/0YEFmhTl25g1CwBJp1aEl4/iwQ+I13Sx5lv/kvLMly6TvOAx8kQdiZJc0pK0np2hhoHerOWIF1d1AgXtkNeJ2v4j3wa4JgFtY1crB5TFWSV+NoKvzZEDm+pD7mr7+IK3rb+1yhcmYaVbZhJyeTx9br6IIYPsmJVXAf1BK+Rfn HFRiOSfs s2y2nrVcIFfGnNcleNhqVOIne3mo5LpV1KbQkJEcLZpVUjCXAhIIlKlw7WaQKx4OQkcxpzCeH3gv+b6MJzJ0v0Z4B9i40YvCcpE6uV0aYAfH7g+yysNO2BVmseVsgJiN6EcMm/VyuFK+3tKi5nyoLaXxijjG26bVmafYNbPAL6UGvStAXzZZoIAvFxArG5dlG57YROHxIFUMi28dKU7hUFt5CJT1eqzadOY3IWY+1rCKQIxikhHDdujzq6n0DKE4qzRxU13lvKAQpwOtGzZ2eAdCAY9yL9p/R/zcTnm+u22gAGAxK0bvefYrXIDfjVkj6cEoavnf5ymFqerytctXH+gnQAseBEJgMJbCVcEj9Q8/GzOZsF6/oUtZm5PTkwRJZTSH4HTIfsG2HNA52xOHjKH4G/7VZq4ovCj6u2LV1XOMSr6CozQ1BLgmvv29wo7E41DHvcONT3HTCz/QajO+omAoNiWGOkM/iDeWhWm9qEblp1cY3ezb0vw0Z4fpXJNDLuLqFGrJzNCIbGzuooLe9SINfN+OkNtBXc5Hwjvtg5hd/PnmOUcZ+3LQMtIHoF1otCFCLbEvbDUNUeRxWE7x2FS3MGRD2AGVSbZO9Bxi9H0L6r0omZ5sJA7OLnzdLQU4dErhZ8P/Lk3+tkFI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With frontswap, zram can perform swapout 13% faster, which restores its original performance with rw_page. 200M swapout using MADV_PAGEOUT: Before: 4355ms After: 3786ms Signed-off-by: Minchan Kim --- drivers/block/zram/Kconfig | 1 + drivers/block/zram/zram_drv.c | 98 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 3 files changed, 100 insertions(+) diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig index 0386b7da02aa..a841d6d14c74 100644 --- a/drivers/block/zram/Kconfig +++ b/drivers/block/zram/Kconfig @@ -4,6 +4,7 @@ config ZRAM depends on BLOCK && SYSFS && MMU depends on CRYPTO_LZO || CRYPTO_ZSTD || CRYPTO_LZ4 || CRYPTO_LZ4HC || CRYPTO_842 select ZSMALLOC + select FRONTSWAP help Creates virtual block devices called /dev/zramX (X = 0, 1, ...). Pages written to these disks are compressed and stored in memory diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5d258a28ec43..5e973c982235 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,8 @@ #include #include #include +#include +#include #include "zram_drv.h" @@ -2170,6 +2172,90 @@ static struct attribute *zram_disk_attrs[] = { ATTRIBUTE_GROUPS(zram_disk); +static struct zram *zram_swaps[MAX_SWAPFILES]; +static LIST_HEAD(zram_list); +static DEFINE_SPINLOCK(zram_list_lock); + +static int zram_frontswap_store(unsigned int type, pgoff_t index, + struct page *page) +{ + int err; + struct zram *zram = zram_swaps[type]; + + err = zram_write_page(zram, page, index); + if (!err) { + zram_slot_lock(zram, index); + zram_accessed(zram, index); + zram_slot_unlock(zram, index); + } + + return err; +} + +static int zram_frontswap_load(unsigned int type, pgoff_t index, + struct page *page, bool *exclusive) +{ + int err; + struct zram *zram = zram_swaps[type]; + + zram_slot_lock(zram, index); + if (zram_test_flag(zram, index, ZRAM_WB)) { + zram_slot_lock(zram, index); + return -1; + } + + err = zram_read_from_zspool(zram, page, index); + if (!err) + zram_accessed(zram, index); + zram_slot_unlock(zram, index); + + return err; +} + +static void zram_frontswap_invalidate_area(unsigned int type) +{ + struct zram *zram = zram_swaps[type]; + + if (!zram) + return; +} + +static void zram_frontswap_init(unsigned int type, struct block_device *bdev) +{ + struct zram *zram; + + spin_lock(&zram_list_lock); + list_for_each_entry(zram, &zram_list, list) { + if (&zram_devops == bdev->bd_disk->fops) { + zram_swaps[type] = zram; + break; + } + } + spin_unlock(&zram_list_lock); +} + +static void zram_frontswap_invalidate_page(unsigned int type, pgoff_t offset) +{ + struct zram *zram = zram_swaps[type]; + + if (!zram_slot_trylock(zram, offset)) { + atomic64_inc(&zram->stats.miss_free); + return; + } + + zram_free_page(zram, offset); + zram_slot_unlock(zram, offset); +} + +static const struct frontswap_ops zram_frontswap_ops = { + .store = zram_frontswap_store, + .load = zram_frontswap_load, + .init = zram_frontswap_init, + .invalidate_area = zram_frontswap_invalidate_area, + .invalidate_page = zram_frontswap_invalidate_page, + .init = zram_frontswap_init +}; + /* * Allocate and initialize new zram device. the function returns * '>= 0' device_id upon success, and negative value otherwise. @@ -2246,6 +2332,10 @@ static int zram_add(void) comp_algorithm_set(zram, ZRAM_PRIMARY_COMP, default_compressor); + spin_lock(&zram_list_lock); + list_add(&zram->list, &zram_list); + spin_unlock(&zram_list_lock); + zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); return device_id; @@ -2303,6 +2393,11 @@ static int zram_remove(struct zram *zram) zram_reset_device(zram); put_disk(zram->disk); + + spin_lock(&zram_list_lock); + list_del(&zram->list); + spin_unlock(&zram_list_lock); + kfree(zram); return 0; } @@ -2428,6 +2523,9 @@ static int __init zram_init(void) num_devices--; } + if (frontswap_register_ops(&zram_frontswap_ops)) + pr_info("Frontswap is not used, which is suboptimal for zram swap.\n"); + return 0; out_error: diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..0f52d2da8512 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -139,5 +139,6 @@ struct zram { #ifdef CONFIG_ZRAM_MEMORY_TRACKING struct dentry *debugfs_dir; #endif + struct list_head list; }; #endif -- 2.41.0.255.g8b1d071c50-goog