From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF1471B040A for ; Mon, 30 Dec 2024 17:46:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735580801; cv=none; b=LN6A/luudsRsOhFJCQRSPTS20vfgsA0Y2OIHORTfS5hWkU9d1qxwHlPyJMw6Ov44b19iNwCBhtccB5+WLExjW2MENchK9YVQPaw3uNkWFu+g55/oEyI1abfOX/iuHo/s/DVuDkIPRKC7EOPh+7qZYkmCS6+l9kBIswPxGyWNoiQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735580801; c=relaxed/simple; bh=0zwLEl1IX5lfCSiyPNTsbs+vpbZuqKYgJ688EZCFVa0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rPJfxPbH7jABW7VpZa1kGq8+0mvlFmuWwf7fiCcx4OENGKmjMy1XEAceBzIjWs7heVhNdj3DIsIWPX8N0+B72QsvM2s8jzkYjmXhQ7ybOS+/d90fGh3lPuHOIYFDYfmmZx/6l5KOkPBH6oe7+ecENiodj/KeJkuQ3kzRfQN/bek= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=b1AdSxxm; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b1AdSxxm" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2ee8aa26415so12457894a91.1 for ; Mon, 30 Dec 2024 09:46:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735580799; x=1736185599; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=b1AdSxxmzoYATyeIFTImv980KLQyFeTZoMryvH/kWRTU/dA4bbtkwRQABYafbEkSjT zV6dawYTTNuTIiqg1O6wxrGkWQKuj1x8oMV6DYJWEoUXROKZHDrIbdvtZcM5x21X4A3+ 3k6pVb4vdbZUv1Jnep2WYl++Yh0b5wAbQwgBLfu+F+2z+lrZlegSPPQM0U3bIS3WhoeR wQZ/y0ooMkGROwRngQEB1Roi4QyPdYAEqGUhHRCn1EZUszsI8DTqKgL0KXTbcaFFSgMC 9Tgxo2TpJ1GjfxmKq7HVQOM1Fk+n8xVznHZiQJ3SuGJyNx640SDz/YQh0cCehCbfbmW4 m2ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735580799; x=1736185599; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=Mgc6qhtG0i7tNq2pXh2PwEaVzZR8UOiPcriwVorZhMQgpA2PVc9jkpc+PF1CinPbch 08rzikwY/ffbADamem7h25YouPQjdj76FpK0yDkgqPOWVVrujhHZJ09CFBCpF/38qSJG 2hnX1qLkMIIW3N07gvRXp+W2eNt5ba/4CvRUxYE5cNzPAv+JZSTdv7v8X9NVrV7yoTIj hYxFqwwcgkXA+rZZZJuZvfTorBdAGZTF6WhoiFlL1dpt2V2X0B8JlAS7ZpFVcjSH/EUS iYMJmPAwCBR97L2w5/vksd9N5KSImQpPkh9Aw478A8gtZ4/iI1TjJPD/498No05ArI15 9Fxg== X-Forwarded-Encrypted: i=1; AJvYcCV3rBaIFxBX7RaAIPu/QPTpf0O3SaAi5FoQyw1y68Kmqxg8JtforiqrzQ60cpSBTZMN49CsgInlPFrartI=@vger.kernel.org X-Gm-Message-State: AOJu0Yzo5lqYyiQRDNsV264s3SFVDl+8sVOvJcHZVfsUpTHVuTi5W1Uv Bk82m+iQA7bUEGX3OZY9zyYuOqEhHg/UXeIwc/imyFpv0rCzbGh1 X-Gm-Gg: ASbGncspuD/TdF8SYfjpNTTLf9SVpR8iPqdp3AEsh+d6dH46fQ4TQdefLvm1NcebH4S js9iNdvTprdMIjgnCzYQ/oaKAz1lYNkNBeskTeMNM14Mpp4o9+V+/Yi/la4LvpJCeIQ+FQwb7Qf G0XJuyozzMxTrKSj/YpU6td0gP6JDCr/2xYQeVqIYDUibgrqd76l7N8LnTvwB6c75Y9zhLuMPvP XfwUMniQziZjQyJBXC2mqdgRlUnp2kc0Tl8rsXgbouXopT9N4zLuLcxTesOLLz+MNyALrxwaYAX X-Google-Smtp-Source: AGHT+IEpwr2sGmtXS7EYPdFpiPU2k4iF8zS23kwgmM7X2KfZ2TC2ulrhiLKT3yI/wIFjfHbUR/OxOA== X-Received: by 2002:a17:90a:d448:b0:2ee:5edc:489 with SMTP id 98e67ed59e1d1-2f452ee6dddmr47363487a91.26.1735580798893; Mon, 30 Dec 2024 09:46:38 -0800 (PST) Received: from KASONG-MC4.tencent.com ([1.203.117.231]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc9cdf25sm180687695ad.118.2024.12.30.09.46.35 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 30 Dec 2024 09:46:38 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v3 01/13] mm, swap: minor clean up for swap entry allocation Date: Tue, 31 Dec 2024 01:46:09 +0800 Message-ID: <20241230174621.61185-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241230174621.61185-1-ryncsn@gmail.com> References: <20241230174621.61185-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Kairui Song Direct reclaim can skip the whole folio after reclaimed a set of folio based slots. Also simplify the code for allocation, reduce indention. Signed-off-by: Kairui Song --- mm/swapfile.c | 59 +++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b0a9071cfe1d..f8002f110104 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -604,23 +604,28 @@ static bool cluster_reclaim_range(struct swap_info_struct *si, unsigned long start, unsigned long end) { unsigned char *map = si->swap_map; - unsigned long offset; + unsigned long offset = start; + int nr_reclaim; spin_unlock(&ci->lock); spin_unlock(&si->lock); - for (offset = start; offset < end; offset++) { + do { switch (READ_ONCE(map[offset])) { case 0: - continue; + offset++; + break; case SWAP_HAS_CACHE: - if (__try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT) > 0) - continue; - goto out; + nr_reclaim = __try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT); + if (nr_reclaim > 0) + offset += nr_reclaim; + else + goto out; + break; default: goto out; } - } + } while (offset < end); out: spin_lock(&si->lock); spin_lock(&ci->lock); @@ -838,35 +843,30 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o &found, order, usage); frags++; if (found) - break; + goto done; } - if (!found) { + /* + * Nonfull clusters are moved to frag tail if we reached + * here, count them too, don't over scan the frag list. + */ + while (frags < si->frag_cluster_nr[order]) { + ci = list_first_entry(&si->frag_clusters[order], + struct swap_cluster_info, list); /* - * Nonfull clusters are moved to frag tail if we reached - * here, count them too, don't over scan the frag list. + * Rotate the frag list to iterate, they were all failing + * high order allocation or moved here due to per-CPU usage, + * this help keeping usable cluster ahead. */ - while (frags < si->frag_cluster_nr[order]) { - ci = list_first_entry(&si->frag_clusters[order], - struct swap_cluster_info, list); - /* - * Rotate the frag list to iterate, they were all failing - * high order allocation or moved here due to per-CPU usage, - * this help keeping usable cluster ahead. - */ - list_move_tail(&ci->list, &si->frag_clusters[order]); - offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), - &found, order, usage); - frags++; - if (found) - break; - } + list_move_tail(&ci->list, &si->frag_clusters[order]); + offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), + &found, order, usage); + frags++; + if (found) + goto done; } } - if (found) - goto done; - if (!list_empty(&si->discard_clusters)) { /* * we don't have free cluster but have some clusters in @@ -904,7 +904,6 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o goto done; } } - done: cluster->next[order] = offset; return found; -- 2.47.1