From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BBB8C54E67 for ; Thu, 21 Mar 2024 03:08:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 031346B0083; Wed, 20 Mar 2024 23:08:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F210C6B0085; Wed, 20 Mar 2024 23:08:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE86B6B0087; Wed, 20 Mar 2024 23:08:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CC8576B0083 for ; Wed, 20 Mar 2024 23:08:04 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 67335A1476 for ; Thu, 21 Mar 2024 03:08:04 +0000 (UTC) X-FDA: 81919562088.23.96407BA Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) by imf07.hostedemail.com (Postfix) with ESMTP id 1F1A840011 for ; Thu, 21 Mar 2024 03:08:01 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=C5ThTTZ3; spf=pass (imf07.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.179 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710990482; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=I4yvMLSyJ52YXPPQpMAAR395HKgpjyDA2W9HqIEfUS0=; b=r542XVAblzAQV8NzXgNMs5v/8BvoMBp94MGBppJ69+z9qeXNTo20o+ggXyOkIIsbpUGHmL uSndV4LZyQlBDad5gqN0ghIzLZnDENeCURZEPjtI3sPoG2inc0vmCiUDROMIwq0sBIcuhi 1ue0vfRCdAamgkb04SPhbl0mwq7YAFs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710990482; a=rsa-sha256; cv=none; b=Fwky9AnzMs3N6stoZ8fFGqGIdswRrB3n6NOO4bYMKI8OqNLAzIfH9Z/QSUW0fWIT4VRMSE 4Em7p0UzAX1NJ2pqsunBG0RJ1ns3HkZ/i47gi7g12gxQdXUrDGtKORWoeebhNH6X5qcESv njQAz6PJ9RSuEi1YGNpNX0BC+XXwsYU= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=C5ThTTZ3; spf=pass (imf07.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.179 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Message-ID: <37243db7-41e3-4d07-a715-6957a0c15912@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1710990479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I4yvMLSyJ52YXPPQpMAAR395HKgpjyDA2W9HqIEfUS0=; b=C5ThTTZ3MpwzELGp2auc6YdvOdgRVj62zL2R3y0OVucEQK8/UlUIxJpgOXeuE4eIPXmOQ6 wHMAX7nivouQ58wJUzIXUI4ixRZIbCFTmlq8urGNR/AWI0OH84cFEz3D8p70+OeHC4Rlku hASh6hHFMDvOZjmwAFIOcmGttc7ZoOU= Date: Thu, 21 Mar 2024 11:07:51 +0800 MIME-Version: 1.0 Subject: Re: [PATCH v8] zswap: replace RB tree with xarray To: Chris Li , Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed , Nhat Pham , Johannes Weiner , "Matthew Wilcox (Oracle)" , Chengming Zhou , Barry Song References: <20240320-zswap-xarray-v8-1-dc08ee9f8744@kernel.org> Content-Language: en-US X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: <20240320-zswap-xarray-v8-1-dc08ee9f8744@kernel.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT X-Stat-Signature: jf7sso9puh5wguwwkma9opmqz796qeyd X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1F1A840011 X-Rspam-User: X-HE-Tag: 1710990481-811962 X-HE-Meta: U2FsdGVkX19XhKvgrLnFVThDmEl8jxC++WRcFlmEnU/kmSFpBWgNnZo8GHsujmmn1/D5trWAT0Q9klql0b5M9QzCIlUyQCeNO7nzX3lt8334nLRjhl3co/fywRFiGCxIXb9UpzZBOMS9vLcbQgjrxWv1g2HRa++rPuKfv9gWB7ntC/eYp5G4VdjBKlLjbNvFbIXazoHXfEY5EM+/qnaM75pzA7J0dPvxtxYpCoPRtuKlGtG/Q7piFb8Iuc2Bh+yvmIrRKPcPPMgIC4RviCAt7BR3zpJfnxnRJnmiNidaP/5ntpVAIn7GnK3AT6dUJDbtjqdIGNCsd4x4VF0iyNzqc2d9Jygj5bkyd9rUUBhI7JyVSMAnR+9mPMoLC3p8xXqAidQTi+UckWMjvq1tec+pASU/MDiFMyndMBKvyanqh4fAxtMA3J292Vmjcy29wlD+nsPlRiZNEQSAgEcQt4qiesb0zI+Vn+xNSoOzeXAGNVeioGHGTINAHU58kRD2uCHAvYdwCxtBizEbD5sXeLH2Hv01nU0memSZjmIsoy0iInuBaWGQBnlOIJ8SyOneYdl4dPj3sw/o97PpxunKxwfAx3d+70vmVttiSDFzlyB3yy0OoF5y5mTYEFYUM8zcVZ1UwiktXprKYX6Ls74NNsLD2vuc4dN+I8Rl/WCcoWUxEGZyB8A8MHzA6adSq8j8uGj+DVZmqhAC48JBN3IPhScEBm1N5M815/IyUs9RcjkJoxnIoK+2Fno8xN24dz8PiXiXFBLnVdMDSyGtziHN277zav6ogOrdOXglWmaBh35HtcINHbScKFIlIDVqMRCcGnfxUAcUGa/KapxU8G87k/t2BqAubSr3xxte23eyrzgtTLl5Hnp7gdHhpJHXLsgTocLuhBm2A9dJzfYmmN7I+Kzt5L1KNcfLkAjPwyq4WvWF0XdizjLYARa9KkMr8GkAC7N9WXguQoLdYVYwanjcKAM ZrHDndCE vLn9YP8dM5+ick02JgTKRuNn5nRxITrgOw+IbkEko2A3DBfW/Qksdl/9Fpprp3LTwuAir0L7W96Iv7JJrcKZ95fnZKK/f5exyA6AGalTC84v20qcn0+i6MMtHOqAV1BfTRtSX4FgQeWdkrfhTvlEAGvsDJsbwDYw/XoLS2NtucMTu5jpcx2u1w+/i5ijhz+8y5EskU9mkoFpOtnTNQAkbFv+PzBQqK+v9eS8drIKJ2of40KxT+fxKNNsz0w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/3/21 03:31, Chris Li wrote: > Very deep RB tree requires rebalance at times. That > contributes to the zswap fault latencies. Xarray does not > need to perform tree rebalance. Replacing RB tree to xarray > can have some small performance gain. > > One small difference is that xarray insert might fail with > ENOMEM, while RB tree insert does not allocate additional > memory. > > The zswap_entry size will reduce a bit due to removing the > RB node, which has two pointers and a color field. Xarray > store the pointer in the xarray tree rather than the > zswap_entry. Every entry has one pointer from the xarray > tree. Overall, switching to xarray should save some memory, > if the swap entries are densely packed. > > Notice the zswap_rb_search and zswap_rb_insert always > followed by zswap_rb_erase. Use xa_erase and xa_store > directly. That saves one tree lookup as well. > > Remove zswap_invalidate_entry due to no need to call > zswap_rb_erase any more. Use zswap_free_entry instead. > > The "struct zswap_tree" has been replaced by "struct xarray". > The tree spin lock has transferred to the xarray lock. > > Run the kernel build testing 10 times for each version, averages: > (memory.max=2GB, zswap shrinker and writeback enabled, > one 50GB swapfile, 24 HT core, 32 jobs) > > mm-unstable-a824831a082f xarray v7 > user 3547.264 3541.509 > sys 531.176 526.111 > real 200.752 201.334 > > Reviewed-by: Nhat Pham > Acked-by: Yosry Ahmed > Acked-by: Johannes Weiner > Signed-off-by: Chris Li Very nice! Reviewed-by: Chengming Zhou Thanks. > --- > Changes in v8: > - Fix some white space damage discovered by Yosry. No actual code change. > - Collect review tags. > - Link to v7: https://lore.kernel.org/r/20240319-zswap-xarray-v7-1-e9a03a049e86@kernel.org > > Changes in v7: > - update comment suggested by Johannes and Yosry > - Simplify some error handling code, suggested by Johannes. > - Link to v6: https://lore.kernel.org/r/20240312-zswap-xarray-v6-1-1b82027d7082@kernel.org > > Changes in v6: > - Add WARN_ONCE() for xa_store failed other than -ENOMEM. > - Collect review tags. > - Link to v5: https://lore.kernel.org/r/20240311-zswap-xarray-v5-1-a3031feb9c85@kernel.org > > Changes in v5: > - Remove zswap_xa_insert(), call xa_store and xa_erase directly. > - Remove zswap_reject_xarray_fail. > - Link to v4: https://lore.kernel.org/r/20240304-zswap-xarray-v4-1-c4b45670cc30@kernel.org > > Changes in v4: > - Remove zswap_xa_search_and_earse, use xa_erase directly. > - Move charge of objcg after zswap_xa_insert. > - Avoid erase old entry on insert fail error path. > - Remove not needed swap_zswap_tree change > - Link to v3: https://lore.kernel.org/r/20240302-zswap-xarray-v3-1-5900252f2302@kernel.org > > Changes in v3: > - Use xa_cmpxchg instead of zswap_xa_search_and_delete in zswap_writeback_entry. > - Use xa_store in zswap_xa_insert directly. Reduce the scope of spinlock. > - Fix xa_store error handling for same page fill case. > - Link to v2: https://lore.kernel.org/r/20240229-zswap-xarray-v2-1-e50284dfcdb1@kernel.org > > Changes in v2: > - Replace struct zswap_tree with struct xarray. > - Remove zswap_tree spinlock, use xarray lock instead. > - Fold zswap_rb_erase() into zswap_xa_search_and_delete() and zswap_xa_insert(). > - Delete zswap_invalidate_entry(), use zswap_free_entry() instead. > - Link to v1: https://lore.kernel.org/r/20240117-zswap-xarray-v1-0-6daa86c08fae@kernel.org > --- > mm/zswap.c | 178 ++++++++++++++++++------------------------------------------- > 1 file changed, 53 insertions(+), 125 deletions(-) > > diff --git a/mm/zswap.c b/mm/zswap.c > index b31c977f53e9..a97de9e547df 100644 > --- a/mm/zswap.c > +++ b/mm/zswap.c > @@ -20,7 +20,6 @@ > #include > #include > #include > -#include > #include > #include > #include > @@ -196,7 +195,6 @@ static struct shrinker *zswap_shrinker; > * This structure contains the metadata for tracking a single compressed > * page within zswap. > * > - * rbnode - links the entry into red-black tree for the appropriate swap type > * swpentry - associated swap entry, the offset indexes into the red-black tree > * length - the length in bytes of the compressed page data. Needed during > * decompression. For a same value filled page length is 0, and both > @@ -208,7 +206,6 @@ static struct shrinker *zswap_shrinker; > * lru - handle to the pool's lru used to evict pages. > */ > struct zswap_entry { > - struct rb_node rbnode; > swp_entry_t swpentry; > unsigned int length; > struct zswap_pool *pool; > @@ -220,12 +217,7 @@ struct zswap_entry { > struct list_head lru; > }; > > -struct zswap_tree { > - struct rb_root rbroot; > - spinlock_t lock; > -}; > - > -static struct zswap_tree *zswap_trees[MAX_SWAPFILES]; > +static struct xarray *zswap_trees[MAX_SWAPFILES]; > static unsigned int nr_zswap_trees[MAX_SWAPFILES]; > > /* RCU-protected iteration */ > @@ -253,7 +245,7 @@ static bool zswap_has_pool; > * helpers and fwd declarations > **********************************/ > > -static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) > +static inline struct xarray *swap_zswap_tree(swp_entry_t swp) > { > return &zswap_trees[swp_type(swp)][swp_offset(swp) > >> SWAP_ADDRESS_SPACE_SHIFT]; > @@ -792,63 +784,6 @@ void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) > spin_unlock(&zswap_shrink_lock); > } > > -/********************************* > -* rbtree functions > -**********************************/ > -static struct zswap_entry *zswap_rb_search(struct rb_root *root, pgoff_t offset) > -{ > - struct rb_node *node = root->rb_node; > - struct zswap_entry *entry; > - pgoff_t entry_offset; > - > - while (node) { > - entry = rb_entry(node, struct zswap_entry, rbnode); > - entry_offset = swp_offset(entry->swpentry); > - if (entry_offset > offset) > - node = node->rb_left; > - else if (entry_offset < offset) > - node = node->rb_right; > - else > - return entry; > - } > - return NULL; > -} > - > -/* > - * In the case that a entry with the same offset is found, a pointer to > - * the existing entry is stored in dupentry and the function returns -EEXIST > - */ > -static int zswap_rb_insert(struct rb_root *root, struct zswap_entry *entry, > - struct zswap_entry **dupentry) > -{ > - struct rb_node **link = &root->rb_node, *parent = NULL; > - struct zswap_entry *myentry; > - pgoff_t myentry_offset, entry_offset = swp_offset(entry->swpentry); > - > - while (*link) { > - parent = *link; > - myentry = rb_entry(parent, struct zswap_entry, rbnode); > - myentry_offset = swp_offset(myentry->swpentry); > - if (myentry_offset > entry_offset) > - link = &(*link)->rb_left; > - else if (myentry_offset < entry_offset) > - link = &(*link)->rb_right; > - else { > - *dupentry = myentry; > - return -EEXIST; > - } > - } > - rb_link_node(&entry->rbnode, parent, link); > - rb_insert_color(&entry->rbnode, root); > - return 0; > -} > - > -static void zswap_rb_erase(struct rb_root *root, struct zswap_entry *entry) > -{ > - rb_erase(&entry->rbnode, root); > - RB_CLEAR_NODE(&entry->rbnode); > -} > - > /********************************* > * zswap entry functions > **********************************/ > @@ -860,7 +795,6 @@ static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) > entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); > if (!entry) > return NULL; > - RB_CLEAR_NODE(&entry->rbnode); > return entry; > } > > @@ -896,17 +830,6 @@ static void zswap_entry_free(struct zswap_entry *entry) > atomic_dec(&zswap_stored_pages); > } > > -/* > - * The caller hold the tree lock and search the entry from the tree, > - * so it must be on the tree, remove it from the tree and free it. > - */ > -static void zswap_invalidate_entry(struct zswap_tree *tree, > - struct zswap_entry *entry) > -{ > - zswap_rb_erase(&tree->rbroot, entry); > - zswap_entry_free(entry); > -} > - > /********************************* > * compressed storage functions > **********************************/ > @@ -1106,7 +1029,8 @@ static void zswap_decompress(struct zswap_entry *entry, struct page *page) > static int zswap_writeback_entry(struct zswap_entry *entry, > swp_entry_t swpentry) > { > - struct zswap_tree *tree; > + struct xarray *tree; > + pgoff_t offset = swp_offset(swpentry); > struct folio *folio; > struct mempolicy *mpol; > bool folio_was_allocated; > @@ -1143,19 +1067,13 @@ static int zswap_writeback_entry(struct zswap_entry *entry, > * be dereferenced. > */ > tree = swap_zswap_tree(swpentry); > - spin_lock(&tree->lock); > - if (zswap_rb_search(&tree->rbroot, swp_offset(swpentry)) != entry) { > - spin_unlock(&tree->lock); > + if (entry != xa_cmpxchg(tree, offset, entry, NULL, GFP_KERNEL)) { > delete_from_swap_cache(folio); > folio_unlock(folio); > folio_put(folio); > return -ENOMEM; > } > > - /* Safe to deref entry after the entry is verified above. */ > - zswap_rb_erase(&tree->rbroot, entry); > - spin_unlock(&tree->lock); > - > zswap_decompress(entry, &folio->page); > > count_vm_event(ZSWPWB); > @@ -1467,8 +1385,8 @@ bool zswap_store(struct folio *folio) > { > swp_entry_t swp = folio->swap; > pgoff_t offset = swp_offset(swp); > - struct zswap_tree *tree = swap_zswap_tree(swp); > - struct zswap_entry *entry, *dupentry; > + struct xarray *tree = swap_zswap_tree(swp); > + struct zswap_entry *entry, *old; > struct obj_cgroup *objcg = NULL; > struct mem_cgroup *memcg = NULL; > unsigned long max_pages, cur_pages; > @@ -1556,28 +1474,44 @@ bool zswap_store(struct folio *folio) > insert_entry: > entry->swpentry = swp; > entry->objcg = objcg; > + > + old = xa_store(tree, offset, entry, GFP_KERNEL); > + if (xa_is_err(old)) { > + int err = xa_err(old); > + > + WARN_ONCE(err != -ENOMEM, "unexpected xarray error: %d\n", err); > + zswap_reject_alloc_fail++; > + goto store_failed; > + } > + > + /* > + * We may have had an existing entry that became stale when > + * the folio was redirtied and now the new version is being > + * swapped out. Get rid of the old. > + */ > + if (old) > + zswap_entry_free(old); > + > if (objcg) { > obj_cgroup_charge_zswap(objcg, entry->length); > - /* Account before objcg ref is moved to tree */ > count_objcg_event(objcg, ZSWPOUT); > } > > - /* map */ > - spin_lock(&tree->lock); > /* > - * The folio may have been dirtied again, invalidate the > - * possibly stale entry before inserting the new entry. > + * We finish initializing the entry while it's already in xarray. > + * This is safe because: > + * > + * 1. Concurrent stores and invalidations are excluded by folio lock. > + * > + * 2. Writeback is excluded by the entry not being on the LRU yet. > + * The publishing order matters to prevent writeback from seeing > + * an incoherent entry. > */ > - if (zswap_rb_insert(&tree->rbroot, entry, &dupentry) == -EEXIST) { > - zswap_invalidate_entry(tree, dupentry); > - WARN_ON(zswap_rb_insert(&tree->rbroot, entry, &dupentry)); > - } > if (entry->length) { > INIT_LIST_HEAD(&entry->lru); > zswap_lru_add(&zswap_list_lru, entry); > atomic_inc(&zswap_nr_stored); > } > - spin_unlock(&tree->lock); > > /* update stats */ > atomic_inc(&zswap_stored_pages); > @@ -1585,8 +1519,14 @@ bool zswap_store(struct folio *folio) > > return true; > > +store_failed: > + if (!entry->length) > + atomic_dec(&zswap_same_filled_pages); > + else { > + zpool_free(zswap_find_zpool(entry), entry->handle); > put_pool: > - zswap_pool_put(entry->pool); > + zswap_pool_put(entry->pool); > + } > freepage: > zswap_entry_cache_free(entry); > reject: > @@ -1597,11 +1537,9 @@ bool zswap_store(struct folio *folio) > * possibly stale entry which was previously stored at this offset. > * Otherwise, writeback could overwrite the new data in the swapfile. > */ > - spin_lock(&tree->lock); > - entry = zswap_rb_search(&tree->rbroot, offset); > + entry = xa_erase(tree, offset); > if (entry) > - zswap_invalidate_entry(tree, entry); > - spin_unlock(&tree->lock); > + zswap_entry_free(entry); > return false; > > shrink: > @@ -1614,20 +1552,15 @@ bool zswap_load(struct folio *folio) > swp_entry_t swp = folio->swap; > pgoff_t offset = swp_offset(swp); > struct page *page = &folio->page; > - struct zswap_tree *tree = swap_zswap_tree(swp); > + struct xarray *tree = swap_zswap_tree(swp); > struct zswap_entry *entry; > u8 *dst; > > VM_WARN_ON_ONCE(!folio_test_locked(folio)); > > - spin_lock(&tree->lock); > - entry = zswap_rb_search(&tree->rbroot, offset); > - if (!entry) { > - spin_unlock(&tree->lock); > + entry = xa_erase(tree, offset); > + if (!entry) > return false; > - } > - zswap_rb_erase(&tree->rbroot, entry); > - spin_unlock(&tree->lock); > > if (entry->length) > zswap_decompress(entry, page); > @@ -1651,19 +1584,17 @@ bool zswap_load(struct folio *folio) > void zswap_invalidate(swp_entry_t swp) > { > pgoff_t offset = swp_offset(swp); > - struct zswap_tree *tree = swap_zswap_tree(swp); > + struct xarray *tree = swap_zswap_tree(swp); > struct zswap_entry *entry; > > - spin_lock(&tree->lock); > - entry = zswap_rb_search(&tree->rbroot, offset); > + entry = xa_erase(tree, offset); > if (entry) > - zswap_invalidate_entry(tree, entry); > - spin_unlock(&tree->lock); > + zswap_entry_free(entry); > } > > int zswap_swapon(int type, unsigned long nr_pages) > { > - struct zswap_tree *trees, *tree; > + struct xarray *trees, *tree; > unsigned int nr, i; > > nr = DIV_ROUND_UP(nr_pages, SWAP_ADDRESS_SPACE_PAGES); > @@ -1673,11 +1604,8 @@ int zswap_swapon(int type, unsigned long nr_pages) > return -ENOMEM; > } > > - for (i = 0; i < nr; i++) { > - tree = trees + i; > - tree->rbroot = RB_ROOT; > - spin_lock_init(&tree->lock); > - } > + for (i = 0; i < nr; i++) > + xa_init(trees + i); > > nr_zswap_trees[type] = nr; > zswap_trees[type] = trees; > @@ -1686,7 +1614,7 @@ int zswap_swapon(int type, unsigned long nr_pages) > > void zswap_swapoff(int type) > { > - struct zswap_tree *trees = zswap_trees[type]; > + struct xarray *trees = zswap_trees[type]; > unsigned int i; > > if (!trees) > @@ -1694,7 +1622,7 @@ void zswap_swapoff(int type) > > /* try_to_unuse() invalidated all the entries already */ > for (i = 0; i < nr_zswap_trees[type]; i++) > - WARN_ON_ONCE(!RB_EMPTY_ROOT(&trees[i].rbroot)); > + WARN_ON_ONCE(!xa_empty(trees + i)); > > kvfree(trees); > nr_zswap_trees[type] = 0; > > --- > base-commit: a824831a082f1d8f9b51a4c0598e633d38555fcf > change-id: 20240104-zswap-xarray-716260e541e3 > > Best regards,