From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-180.mta1.migadu.com (out-180.mta1.migadu.com [95.215.58.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72EF93CBE71 for ; Tue, 7 Jul 2026 07:33:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783409632; cv=none; b=DOtl6sscZdbF8Eli4y7L8zh7OXHfLT4cn/ckZb/acg0FYu3KNdK8WleVufMuKQ50hXnAVHFky9z5qNa60SOoHoBiTEvrUICU4QuMvPkrC0s1Sfi6DF1QB9VuW38z+kEQQ6i5gZcp+N5leVNxEoBKT8EPzGg0yMRoIQSKqpsi/hY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783409632; c=relaxed/simple; bh=VC8dXPKIi0cX7/d7reDDLN1bVXnp5IjL3W+6YGiWvrs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-type; b=Sqz/3blvvbM/pft6VS18RK90xfP1DHAPBcuxTdD7awN+laUmddS5YwDiLl112ueStaroTiNxxskM2QElRXYTSQa8+cQf+oxlI4aLuya4+SoVMdBbZvszyxQ6yZUvwiSmOtNY9QTFtL459MkbqTl0vyt0ehlRPySbC5h/wP1jr4E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=NuXKu0DZ; arc=none smtp.client-ip=95.215.58.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="NuXKu0DZ" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1783409627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKVZ/psH/tEHHl32AP1GNPm3Y5x3NGt0wvhWlFRYJvg=; b=NuXKu0DZjUo5DrsrUYwbSM9Rd5esEWdalKeFYSU3MAWRKu2Llq00qlxcNF7Z9c/kynDBYP a1ObF7hnihCG/QbQqZG9BEB0n3laKfDhBbfTpMpEdRPEHf6gajDcyjNSA6zQE/S0n3aFcw sEv/zZ4V3sYOnMvd8Ao4Ntsc1cJlDzA= From: Baoquan He To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, yosry@kernel.org, nphamcs@gmail.com, chengming.zhou@linux.dev, chrisl@kernel.org, kasong@tencent.com, shikemeng@huaweicloud.com, baohua@kernel.org, youngjun.park@lge.com, linux-kernel@vger.kernel.org, Baoquan He Subject: [RFC PATCH 5/7] mm/swapfile: handle Pointer entries in count/dup/put paths Date: Tue, 7 Jul 2026 15:32:02 +0800 Message-ID: <20260707073215.72183-6-baoquan.he@linux.dev> In-Reply-To: <20260707073215.72183-1-baoquan.he@linux.dev> References: <20260707073215.72183-1-baoquan.he@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-type: text/plain Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT Update swap count, duplicate, and put operations to handle Pointer-type swap table entries where the count is stored in zswap_entry->swp_tb_val rather than inline in the entry bits. Signed-off-by: Baoquan He --- mm/swapfile.c | 95 ++++++++++++++++++++++++++++++++++++++++----------- mm/zswap.c | 5 +++ 2 files changed, 81 insertions(+), 19 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index a602e5820513..15572383cafa 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1536,25 +1536,51 @@ static void __swap_cluster_put_entry(struct swap_cluster_info *ci, lockdep_assert_held(&ci->lock); swp_tb = __swap_table_get(ci, ci_off); + + /* Pointer entries store count in the zswap_entry struct */ + if (swp_tb_is_pointer(swp_tb)) { + zswap_swp_tb_put_count(swp_tb, ci, ci_off); + return; + } + count = __swp_tb_get_count(swp_tb); VM_WARN_ON_ONCE(count <= 0); VM_WARN_ON_ONCE(count > SWP_TB_COUNT_MAX); - - if (count == SWP_TB_COUNT_MAX) { - count = ci->extend_table[ci_off]; - /* Overflow starts with SWP_TB_COUNT_MAX */ - VM_WARN_ON_ONCE(count < SWP_TB_COUNT_MAX); - count--; - if (count == (SWP_TB_COUNT_MAX - 1)) { - ci->extend_table[ci_off] = 0; - __swap_table_set(ci, ci_off, __swp_tb_mk_count(swp_tb, count)); + if (count == SWP_TB_COUNT_MAX) { + if (!ci->extend_table) { + /* + * MAX without extend_table: MAX is the real + * count. Decrement and write back inline. + */ + count--; + __swap_table_set(ci, ci_off, + __swp_tb_mk_count(swp_tb, count)); + } else { + /* + * MAX with extend_table: real count lives in + * ci->extend_table[ci_off]. Pull, decrement + * it, and either move it back inline (if now + * fits) or write it back to the extend table. + */ + count = ci->extend_table[ci_off]; + /* Overflow starts with SWP_TB_COUNT_MAX */ + VM_WARN_ON_ONCE(count < SWP_TB_COUNT_MAX); + count--; + if (count == (SWP_TB_COUNT_MAX - 1)) { + ci->extend_table[ci_off] = 0; + __swap_table_set(ci, ci_off, + __swp_tb_mk_count(swp_tb, count)); + } else { + ci->extend_table[ci_off] = count; + } + } } else { - ci->extend_table[ci_off] = count; + /* count < MAX: simple inline decrement */ + count--; + __swap_table_set(ci, ci_off, + __swp_tb_mk_count(swp_tb, count)); } - } else { - __swap_table_set(ci, ci_off, __swp_tb_mk_count(swp_tb, --count)); - } /* * `SWP_TB_COUNT_MAX - 1` triggers extend table allocation. If the @@ -1596,7 +1622,18 @@ static void swap_put_entries_cluster(struct swap_info_struct *si, ci_end = ci_off + nr; do { swp_tb = __swap_table_get(ci, ci_off); - if (swp_tb_get_count(swp_tb) == 1) { + /* + * Pointer entries store count in the zswap_entry; + * handle them before swp_tb_get_count (which returns + * -EINVAL for non-countable types). + */ + if (swp_tb_is_pointer(swp_tb)) { + if (zswap_swp_tb_get_count(swp_tb) == 1) { + if (ci_batch == -1) + ci_batch = ci_off; + continue; + } + } else if (swp_tb_get_count(swp_tb) == 1) { /* count == 1 and non-cached slots will be batch freed. */ if (!swp_tb_is_folio(swp_tb)) { if (ci_batch == -1) @@ -1647,6 +1684,11 @@ static int __swap_cluster_dup_entry(struct swap_cluster_info *ci, /* Bad or special slots can't be handled */ if (WARN_ON_ONCE(swp_tb_is_bad(swp_tb))) return -EINVAL; + + /* Pointer entries store count in the zswap_entry struct */ + if (swp_tb_is_pointer(swp_tb)) + return zswap_swp_tb_dup_count(swp_tb, ci, ci_off); + count = __swp_tb_get_count(swp_tb); /* Must be either cached or have a count already */ if (WARN_ON_ONCE(!count && !swp_tb_is_folio(swp_tb))) @@ -1923,13 +1965,19 @@ void __swap_cluster_free_entries(struct swap_info_struct *si, old_tb = __swap_table_get(ci, ci_off); /* * Freeing is done after release of the last swap count - * ref, or after swap cache is dropped + * ref, or after swap cache is dropped. A Pointer entry + * means zswap has a compressed copy; the xarray still + * holds the entry and zswap_invalidate will free it. */ - VM_WARN_ON(!swp_tb_is_shadow(old_tb) || __swp_tb_get_count(old_tb) > 1); + if (swp_tb_is_pointer(old_tb)) + VM_WARN_ON(zswap_swp_tb_get_count(old_tb) > 1); + else + VM_WARN_ON(!swp_tb_is_shadow(old_tb) || + __swp_tb_get_count(old_tb) > 1); /* Resetting the slot to NULL also clears the inline flags. */ __swap_table_set(ci, ci_off, null_to_swp_tb()); - if (!SWAP_TABLE_HAS_ZEROFLAG) + if (!SWAP_TABLE_HAS_ZEROFLAG && !swp_tb_is_pointer(old_tb)) __swap_table_clear_zero(ci, ci_off); /* @@ -1961,8 +2009,11 @@ int __swap_count(swp_entry_t entry) { struct swap_cluster_info *ci = __swap_entry_to_cluster(entry); unsigned int ci_off = swp_cluster_offset(entry); + unsigned long swp_tb = __swap_table_get(ci, ci_off); - return swp_tb_get_count(__swap_table_get(ci, ci_off)); + if (swp_tb_is_pointer(swp_tb)) + return zswap_swp_tb_get_count(swp_tb); + return swp_tb_get_count(swp_tb); } /** @@ -1980,6 +2031,8 @@ bool swap_entry_swapped(struct swap_info_struct *si, swp_entry_t entry) swp_tb = swap_table_get(ci, offset % SWAPFILE_CLUSTER); swap_cluster_unlock(ci); + if (swp_tb_is_pointer(swp_tb)) + return zswap_swp_tb_get_count(swp_tb) > 0; return swp_tb_get_count(swp_tb) > 0; } @@ -2556,8 +2609,12 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, if (!folio) { swp_tb = swap_table_get(__swap_entry_to_cluster(entry), swp_cluster_offset(entry)); - if (swp_tb_get_count(swp_tb) <= 0) + if (swp_tb_is_pointer(swp_tb)) { + if (zswap_swp_tb_get_count(swp_tb) <= 0) + continue; + } else if (swp_tb_get_count(swp_tb) <= 0) { continue; + } return -ENOMEM; } diff --git a/mm/zswap.c b/mm/zswap.c index 1bb484d1ac87..58259673ce2b 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1774,6 +1774,11 @@ void zswap_swp_tb_put_count(unsigned long swp_tb, VM_WARN_ON_ONCE(count == 0); if (count == SWP_TB_COUNT_MAX) { + if (!ci->extend_table) { + count--; + entry->swp_tb_val = __swp_tb_mk_count(entry->swp_tb_val, count); + return; + } count = ci->extend_table[ci_off]; /* Overflow starts with SWP_TB_COUNT_MAX */ VM_WARN_ON_ONCE(count < SWP_TB_COUNT_MAX); -- 2.54.0