From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65378C48BF6 for ; Mon, 4 Mar 2024 08:14:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F18196B00A1; Mon, 4 Mar 2024 03:14:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EC7F96B00A2; Mon, 4 Mar 2024 03:14:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D90636B00A3; Mon, 4 Mar 2024 03:14:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C6B0E6B00A1 for ; Mon, 4 Mar 2024 03:14:38 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A54C5A098D for ; Mon, 4 Mar 2024 08:14:38 +0000 (UTC) X-FDA: 81858645036.21.93F7DDB Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf06.hostedemail.com (Postfix) with ESMTP id E2F57180007 for ; Mon, 4 Mar 2024 08:14:36 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kNiKDwTq; spf=pass (imf06.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709540077; a=rsa-sha256; cv=none; b=E87m49EtNFXtiJtWC0rh1JB8bZx90YOmC/U5/9E6YDh2XjdCT+t5EBU+yUXT8TCH97Q0xK bRluejh0e4qqKEuq6fG0O3V4E5zIxhiN0BbwgBr1JU/JPForuxnvmpS28lY+pTg6aWQQh7 xdjYWZ/4HqnF1GZiC98a06I82Il+T7w= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kNiKDwTq; spf=pass (imf06.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709540077; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AOQhq+D2hctSN3bqux7faSvf1ZbbOD2cN4aBjG+5ulk=; b=2KN4Ac7kgVdIMUFcvETJSuYQhojPqNJ1C9sabN0PFIAxmWrt9sTOCcamEKmZTQpCyHizO9 BAVHcepYJ/QDiZcivAp1TILzO5MW82idd1WOsDG/ggNEomlCLfH6O/FwdAamxNHc9ZVhwp DgKR8Y1JgZFn75kosoFvJSrmx68iZR8= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6e4560664b5so3374327b3a.1 for ; Mon, 04 Mar 2024 00:14:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709540076; x=1710144876; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AOQhq+D2hctSN3bqux7faSvf1ZbbOD2cN4aBjG+5ulk=; b=kNiKDwTqVTmiFFDr9MTniw9+1u4Jjh4g/B0+ADcpk49ee891FXtSbgbRO4P+1eNoYL xhhi7ONkoLTO/OvMq8dU7q1RZAhrb06W3OHqTGq3IGusiTIzHTePOm6ZwLp+kX7Y99fm IzGM9+jm4H/Yt0sPLm3GmxJolADgnNSHoOSzMhbls4QAKGJ2JbA1eBStPIQVpkjmUZjc Dou7uuL0kwee/4JUSXGd2u5YjlDlGndcyFi4rG4OdxL7ObG51qLoqgWaQgNkfQu5XEpp UP2FnEXEHLidBSgnBUhj3aHATKgWGsC6SSdIP7HgRI8m7Hrynn6vpEc9kPy/wZtMZ/yG EBvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709540076; x=1710144876; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AOQhq+D2hctSN3bqux7faSvf1ZbbOD2cN4aBjG+5ulk=; b=ChNx4H1Pg9/wW4DAflcmH0OOw7TFqrxl+F0Opljlx8sHaKhmLs0SHibzYXu2v0h0zx 8Or5pMqcsfwUBN9kHNs6NEGYBJY1zODAHpbccFkx1buUPjpAlkNLkR975Xp5449E2WhS abMgZ3GLlGy6iy6o9S2uTbh9Dw4ewpytu3hpZ9Oq6EiCc7qYL/TKM+VSun7/ZNlALDsC 1JtYXzeoxTPcvqq23na1Q6vruMQxc1JrSFcMRQbFdGPf1vK+zjzgBj0uhH0JydIZQbOe S0Qq052I4aRqv1ZK/OTTDCdlmy/mTPVi/908EuxtCbIXgzuTaivkmCOYDnt8FSgChIPt 1Fsg== X-Forwarded-Encrypted: i=1; AJvYcCWFjSpyf+O0mmtN7lYrhPk3emsqvmCKd87wqekQxs8+PpPsY/WGnk5cQgN5WFhwObwdEx6B1nIjm8xyZxMnmz4lEac= X-Gm-Message-State: AOJu0Yxje5/WOpZawXX9XEe9CgSAmmt+pW1raUfxdfgpb+rLzRZsEHMu rbHLC08+pR1th4DkV9S4ZrNrWfQPykmPqs00EUW2ojwNJ/WbNTzN X-Google-Smtp-Source: AGHT+IGEsGkarNyt9Jp8whgPA0poRrIOWMKKVHKbcHA3ADNFk6EzEEaBlL35RANkuwLw0QLGLyniCQ== X-Received: by 2002:a05:6a20:979b:b0:1a1:3000:ab67 with SMTP id hx27-20020a056a20979b00b001a13000ab67mr9356185pzc.46.1709540075740; Mon, 04 Mar 2024 00:14:35 -0800 (PST) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id ka42-20020a056a0093aa00b006e558a67374sm6686387pfb.0.2024.03.04.00.14.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Mar 2024 00:14:35 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org, ryan.roberts@arm.com Cc: chengming.zhou@linux.dev, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, kasong@tencent.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, mhocko@suse.com, nphamcs@gmail.com, shy828301@gmail.com, steven.price@arm.com, surenb@google.com, wangkefeng.wang@huawei.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, Chuanhua Han , Barry Song Subject: [RFC PATCH v3 2/5] mm: swap: introduce swap_nr_free() for batched swap_free() Date: Mon, 4 Mar 2024 21:13:45 +1300 Message-Id: <20240304081348.197341-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240304081348.197341-1-21cnbao@gmail.com> References: <20240304081348.197341-1-21cnbao@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E2F57180007 X-Stat-Signature: fekxkzp386x7geofedpmg8rn66omr36t X-Rspam-User: X-HE-Tag: 1709540076-232089 X-HE-Meta: U2FsdGVkX1+BMZSB9gTuwQPq9EoHif2jUPIxUpNuFUw/2CPT/YliwPkl7/01ekF/B7X/oADVzxzU3R3usVGmtxkB5wvybgvVFiJSwo1UvbpVvi62H1YFQwF85XJVtCe9Iosc7cEpJ9fFuRox18fvQj57tuHaXvS6CBpBhOizVECL0ywOpAM7T/WgEdIaW8X4pZgtvEG664/bL18iwYp9XcbS4H1uzkNbbQmZOtyCheC+Ok9zrUI+pmz0PG4RI+RoXuJ5jio8yjB5gDd/jEiAetskQ522QCQ7GeTAUfuuOP5Up2aaIAlsvz9zi8YyDA0Lu/+YGIQYHRTmyRs7Uf33PmzaWk+oqcbUNYdCNxU6QoO9K/WPzHRoejPZT6hTsUTeDgWWHVGXck49097t7dB+R2TkGn111phOIedr+Gxvq9InH9N5NuBw8gG0tZHinYkxE8ETFx3k01VKSeuKuT2Ad7PCcYOjXR2F+wosCaE/xy664vTA/GHU4iO0wna7qrneRJyOqsqpmLYoUzLUsK4dMNykdEEusS3Hmt777/RszcwjXFdfq3Xw4eqIeS+nO3OOI84pgDufIoWVMh9kKD+rG4DIjG3KvFiNL3T7pB4DN+ICPmlqC5TyfA6wLTQsOzFnCEYoAY4MqLjb6i6AgFCtYBFB0aCQVGYsFU2ehJ5s+60ucMRpV/kZvYdwvwDJzfYECesYKeC6QCddcOhrTBtGRN3qRbHuvLFrIzPxrxICZc+Yr1gAEnu9WmAMoDDA6vnbtTX5T4hTNEJccLtMSp5s3kvTYIC7A3OGPqswngb/T6vEjV9ZXqrLw46cScyk/72xnej1jCYfNSifPPkddzYkcf1HZVk/XMeW/5pVsLqATJKjDV+WdXw0BhbeTXKNFxdVQnAL0X9AKV7AJFwqyFbo2YAOCZdgFaC/Ya4OzK/3pe5IkphVQI7zTqJuzIAYSLf1HK8AWBwFEbTW1UXXmV2 marB+20S SMCkE6uWkRIascfWIkoh/QLZugdq55R10VsaL+CduUfwPpHEofCg1m6OrfbKIL8DWzc/j8pHqDqRvvmmozHuVRUoKZ73ju5GObqmqDsarMM8kDl+ww9VLBkKc7gI2GoyTqHejJuQN6aYt4IW+IgVMalKtu8zKntioqUs2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Chuanhua Han While swapping in a large folio, we need to free swaps related to the whole folio. To avoid frequently acquiring and releasing swap locks, it is better to introduce an API for batched free. Signed-off-by: Chuanhua Han Co-developed-by: Barry Song Signed-off-by: Barry Song --- include/linux/swap.h | 6 ++++++ mm/swapfile.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index 2955f7a78d8d..d6ab27929458 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -481,6 +481,7 @@ extern void swap_shmem_alloc(swp_entry_t); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free(swp_entry_t); +extern void swap_nr_free(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); extern int free_swap_and_cache(swp_entry_t); int swap_type_of(dev_t device, sector_t offset); @@ -561,6 +562,11 @@ static inline void swap_free(swp_entry_t swp) { } +void swap_nr_free(swp_entry_t entry, int nr_pages) +{ + +} + static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } diff --git a/mm/swapfile.c b/mm/swapfile.c index 3f594be83b58..244106998a69 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1341,6 +1341,41 @@ void swap_free(swp_entry_t entry) __swap_entry_free(p, entry); } +/* + * Called after swapping in a large folio, batched free swap entries + * for this large folio, entry should be for the first subpage and + * its offset is aligned with nr_pages + */ +void swap_nr_free(swp_entry_t entry, int nr_pages) +{ + int i; + struct swap_cluster_info *ci; + struct swap_info_struct *p; + unsigned type = swp_type(entry); + unsigned long offset = swp_offset(entry); + DECLARE_BITMAP(usage, SWAPFILE_CLUSTER) = { 0 }; + + /* all swap entries are within a cluster for mTHP */ + VM_BUG_ON(offset % SWAPFILE_CLUSTER + nr_pages > SWAPFILE_CLUSTER); + + if (nr_pages == 1) { + swap_free(entry); + return; + } + + p = _swap_info_get(entry); + + ci = lock_cluster(p, offset); + for (i = 0; i < nr_pages; i++) { + if (__swap_entry_free_locked(p, offset + i, 1)) + __bitmap_set(usage, i, 1); + } + unlock_cluster(ci); + + for_each_clear_bit(i, usage, nr_pages) + free_swap_slot(swp_entry(type, offset + i)); +} + /* * Called after dropping swapcache to decrease refcnt to swap entries. */ -- 2.34.1