From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD4B5C3DA63 for ; Fri, 26 Jul 2024 05:47:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 149146B008C; Fri, 26 Jul 2024 01:47:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F9C26B0092; Fri, 26 Jul 2024 01:47:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0C196B0093; Fri, 26 Jul 2024 01:47:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CCCB56B008C for ; Fri, 26 Jul 2024 01:47:18 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 39C2FA1574 for ; Fri, 26 Jul 2024 05:47:18 +0000 (UTC) X-FDA: 82380820956.26.4521ACD Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf25.hostedemail.com (Postfix) with ESMTP id 917A7A0019 for ; Fri, 26 Jul 2024 05:47:15 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OLTREvHU; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of chrisl@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721972780; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B3AeIbamR5zLeSXaF7RD1RT+KgbNGxQ/Wjs8mg4mW44=; b=tCPjsMD4fk8IFpxtIdRYadjXUTR2okP0NJkqYqiCvmN+S+h/a3V7XwVnBDp3wmLSM1XfAN +FFo7sTJetaAHUR9q1RSdiu5qE7XZpzG3BaFFQXN+RO/CG9rriIYn6ltgmi0tWDVld9+CT DvTj6q4933SjJ1Jytze1IrfoaSTQMzg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721972780; a=rsa-sha256; cv=none; b=c1P/zYrESboUW1J4MmjKQ3EhKm+7zQtord5FgUy5Cui219lve+MqisaPPrU0FTL5ZtuSuQ vPqUr19EgSFLQmoFywtffC/r7SJ0Sq26LWGU8kaFdkJ+UEqYFSoXuZ2k44M7JhnkUZE8Fg jEBJ92ZsV5fhv2z0NVl+CReTc9Sl6ZI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OLTREvHU; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of chrisl@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=chrisl@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id CDECBCE14E4 for ; Fri, 26 Jul 2024 05:47:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B0CEAC4AF09 for ; Fri, 26 Jul 2024 05:47:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721972830; bh=jzXRQdZILYBiNWgUjKvYd4TcxNu6JVgcbdTV+F5qY90=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=OLTREvHUqn8M7BF5YjpTrJI0B7sSf4rrTSTHZHCNKTt5R0nNdCejehKrzOTWWRZG/ gpTPatS3J5oQENvlsVC/H3uDCMSKFVqji1XodK9+TUEf8T4ZkrQE2vPMraE25XSxc/ Bpvq/agwGPbCXtdmJ3HT4v5LeenSDYQFzuqsM6Du5sj88ZDfXyfKXdVo2GaF8cI62o JlYgl+Y+9O9q0nfUgGdetQcxGefuO5X3wulQD4fwWoWPm/9kHkokcwCk+R1h3KFtfv WUklyVYKEWU2CEeumePqxg3ygehpMk4ZUOMBs5IZKdFwQkIqqzTtjOPPx/f7m9Sxct /UwA8XztFstPQ== Received: by mail-yw1-f173.google.com with SMTP id 00721157ae682-6510c0c8e29so16129027b3.0 for ; Thu, 25 Jul 2024 22:47:10 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCWlECB7cUu3BxTGuZQwqWR4N68yZRbs2e/8IdW6Tb5FRgNyHxmy5v5fh59KSsiaYXC7RH+I/y2cvojHV+mxswbDoNE= X-Gm-Message-State: AOJu0YytwJUo/vGkfUvpxmpv9kkWflMcWAvEUhnsf73uTDknxZyhdeUU 0eEy9taQm5jr/Eah9MDvfOo2ocaaRsH70kWhpF9XGXyTriQhkTGKGde/4xGM9+oERP/KPh3ciAM eTJ5B9fcqh0XPLpCzInV8v80+BAQWzkkN5HjXNA== X-Google-Smtp-Source: AGHT+IGZXwmARv4KcIzaeE7p/e4xS4n17b1cEa/WIcBrWNhDRn5dC49mxeEnC5kRB/94lLkTqy/URdXO8G8jJqRRXZI= X-Received: by 2002:a81:ab4d:0:b0:62c:f82b:553f with SMTP id 00721157ae682-67514c47c2emr53368467b3.31.1721972829909; Thu, 25 Jul 2024 22:47:09 -0700 (PDT) MIME-Version: 1.0 References: <20240711-swap-allocator-v4-0-0295a4d4c7aa@kernel.org> <20240711-swap-allocator-v4-1-0295a4d4c7aa@kernel.org> <878qxzxlkv.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <878qxzxlkv.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Chris Li Date: Thu, 25 Jul 2024 22:46:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 1/3] mm: swap: swap cluster switch to double link list To: "Huang, Ying" Cc: Andrew Morton , Kairui Song , Hugh Dickins , Ryan Roberts , Kalesh Singh , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 917A7A0019 X-Stat-Signature: phjgg7cp1s441khfxf3i8rborbp9g69y X-Rspam-User: X-HE-Tag: 1721972835-935067 X-HE-Meta: U2FsdGVkX197kt+832J/dvtcDVqeMmpVA5H5Prr4oP9TOc7CmuPv4Sm7abitELNoL9D3T9Ss22irtZVmsSEtSu+PSpVG9/MWadauG9xcSSBo7fhP/bWMkocMumWx91+jYcUYYui17/RepErNg1ouuSbEBuGxY5z1QBrp9To2N3BqkBPHIXq5n8KQP6rp0StFxCrTg4itSabEQ8HsQG44NsFeLWnERCMJIPgGybRVYw04jm+B7pn3VTwS+Wz70bs9M+VoW5tNAzKf7PiDfxN8sf0rCRdquluFak1Fmol7s9o+L9bra9ddb0iNcFDlDNNhHQ5cKZTAt1eFTofQGX6shsTIHKes1AQOC6h70UI1xKgFSOs84IHuXLnktAfzr8iVEZbgiQ5yDt0GLfbEJdRu1jMNJziX8a9ZRR4zyeSI6fp3SZ6Q0VsqQIcBT305cu/4oEhi1iVmGifctUEBW9Xf9/t523hZbNROf+RDiCvhLFqt5Q7u8ptk5TJAR+4DCytbpwXEJycLHuP/QwiFqdqkPUK1Qvq0u6YLOpESsHxqFHSS+AilSHhZpKwnPKwkQAUNNCLPx0lJfkua7LGsUqWf4m2+7BSe1WKr716vXyFqVeSUQRWi75Bq2pGi4FQF+zCDE5VZMGVGNUMK4A0eJ+H5W6rkscv1GhbTKo+FTM8ivjULtQhc+qtFFMR9pctmwQbsfqLYEVsH9AJOTV4yXBsfkAPfuoi3xTmBox96AJyH2y32yT3nB0B+SCGIdUFyZL2qfpIfbWQ3LipEeFXl/OkNebPFlw6hI34Z6EB+oo0SzAnq9vpztg7uKn5wx+CTovhF0ImoDE321/geP9phWI0pXx/R8xBYw7EIJkemd+GsA0+uXEG2w+uramEZCL6agNownlqXBfED1byFHr0l7hXfNMo7FagElZS06pMZHuvIy727xaTkXOf1ee+uAVQFlK2WQuKvTRESmnwZBkS8R7h 1Spc/eHu J4I9Moiarvjw+mmHKWg+riSyjZXj0cGfOgxMASbpd7FlXn/NEZWvZdBZ9o0lmfh7WmQG2w9pWg1dKZ+dyvc73G+UkUm7lcNV6BBgXXefsRKXJSpY2kOLeDeBavQd0El8eqlqYba9PJIBa9Ud2whJ5bONTgls65hxK4mmhf/ykiZvYUHMwR3X7C9zgDJWR4D4STYjP/H5lbC1kdCTXeQhRUpY2pTGGBUWFMG25XpreAvZpyppwPZasE8Bh7ZFzFI4eGBXbZ6MaOwxdy9dffHwbSDJQC6pd+94VN4jOHaeYDj8FGxmZ4jUn5N9hTpBbiwl+uLs6GGKheGWrLFw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jul 17, 2024 at 11:29=E2=80=AFPM Huang, Ying = wrote: > > Chris Li writes: > > > Previously, the swap cluster used a cluster index as a pointer > > to construct a custom single link list type "swap_cluster_list". > > The next cluster pointer is shared with the cluster->count. > > It prevents puting the non free cluster into a list. > > > > Change the cluster to use the standard double link list instead. > > This allows tracing the nonfull cluster in the follow up patch. > > That way, it is faster to get to the nonfull cluster of that order. > > > > Remove the cluster getter/setter for accessing the cluster > > struct member. > > > > The list operation is protected by the swap_info_struct->lock. > > > > Change cluster code to use "struct swap_cluster_info *" to > > reference the cluster rather than by using index. That is more > > consistent with the list manipulation. It avoids the repeat > > adding index to the cluser_info. The code is easier to understand. > > > > Remove the cluster next pointer is NULL flag, the double link > > list can handle the empty list pretty well. > > > > The "swap_cluster_info" struct is two pointer bigger, because > > 512 swap entries share one swap struct, it has very little impact > ~~~~ > swap_cluster_info ? Did not see this email earlier. Done. > > > on the average memory usage per swap entry. For 1TB swapfile, the > > swap cluster data structure increases from 8MB to 24MB. > > > > Other than the list conversion, there is no real function change > > in this patch. > > > > Signed-off-by: Chris Li > > --- > > include/linux/swap.h | 26 +++--- > > mm/swapfile.c | 225 ++++++++++++++-----------------------------= -------- > > 2 files changed, 70 insertions(+), 181 deletions(-) > > > > diff --git a/include/linux/swap.h b/include/linux/swap.h > > index e473fe6cfb7a..e9be95468fc7 100644 > > --- a/include/linux/swap.h > > +++ b/include/linux/swap.h > > @@ -243,22 +243,21 @@ enum { > > * free clusters are organized into a list. We fetch an entry from the= list to > > * get a free cluster. > > * > > - * The data field stores next cluster if the cluster is free or cluste= r usage > > - * counter otherwise. The flags field determines if a cluster is free.= This is > > - * protected by swap_info_struct.lock. > > + * The flags field determines if a cluster is free. This is > > + * protected by cluster lock. > > */ > > struct swap_cluster_info { > > spinlock_t lock; /* > > * Protect swap_cluster_info fields > > - * and swap_info_struct->swap_map > > - * elements correspond to the swap > > - * cluster > > + * other than list, and swap_info_struct-= >swap_map > > + * elements correspond to the swap cluste= r. > > */ > > - unsigned int data:24; > > - unsigned int flags:8; > > + u16 count; > > + u8 flags; > > + struct list_head list; > > }; > > #define CLUSTER_FLAG_FREE 1 /* This cluster is free */ > > -#define CLUSTER_FLAG_NEXT_NULL 2 /* This cluster has no next cluster *= / > > + > > > > /* > > * The first page in the swap file is the swap header, which is always= marked > > @@ -283,11 +282,6 @@ struct percpu_cluster { > > unsigned int next[SWAP_NR_ORDERS]; /* Likely next allocation offs= et */ > > }; > > > > -struct swap_cluster_list { > > - struct swap_cluster_info head; > > - struct swap_cluster_info tail; > > -}; > > - > > /* > > * The in-memory structure used to track swap areas. > > */ > > @@ -301,7 +295,7 @@ struct swap_info_struct { > > unsigned char *swap_map; /* vmalloc'ed array of usage coun= ts */ > > unsigned long *zeromap; /* vmalloc'ed bitmap to track zer= o pages */ > > struct swap_cluster_info *cluster_info; /* cluster info. Only for= SSD */ > > - struct swap_cluster_list free_clusters; /* free clusters list */ > > + struct list_head free_clusters; /* free clusters list */ > > unsigned int lowest_bit; /* index of first free in swap_ma= p */ > > unsigned int highest_bit; /* index of last free in swap_map= */ > > unsigned int pages; /* total of usable pages of swap = */ > > @@ -332,7 +326,7 @@ struct swap_info_struct { > > * list. > > */ > > struct work_struct discard_work; /* discard worker */ > > - struct swap_cluster_list discard_clusters; /* discard clusters li= st */ > > + struct list_head discard_clusters; /* discard clusters list */ > > struct plist_node avail_lists[]; /* > > * entries in swap_avail_heads,= one > > * entry per node. > > diff --git a/mm/swapfile.c b/mm/swapfile.c > > index f7224bc1320c..f70d25005d2c 100644 > > --- a/mm/swapfile.c > > +++ b/mm/swapfile.c > > @@ -290,62 +290,15 @@ static void discard_swap_cluster(struct swap_info= _struct *si, > > #endif > > #define LATENCY_LIMIT 256 > > > > -static inline void cluster_set_flag(struct swap_cluster_info *info, > > - unsigned int flag) > > -{ > > - info->flags =3D flag; > > -} > > - > > -static inline unsigned int cluster_count(struct swap_cluster_info *inf= o) > > -{ > > - return info->data; > > -} > > - > > -static inline void cluster_set_count(struct swap_cluster_info *info, > > - unsigned int c) > > -{ > > - info->data =3D c; > > -} > > - > > -static inline void cluster_set_count_flag(struct swap_cluster_info *in= fo, > > - unsigned int c, unsigned int f) > > -{ > > - info->flags =3D f; > > - info->data =3D c; > > -} > > - > > -static inline unsigned int cluster_next(struct swap_cluster_info *info= ) > > -{ > > - return info->data; > > -} > > - > > -static inline void cluster_set_next(struct swap_cluster_info *info, > > - unsigned int n) > > -{ > > - info->data =3D n; > > -} > > - > > -static inline void cluster_set_next_flag(struct swap_cluster_info *inf= o, > > - unsigned int n, unsigned int f) > > -{ > > - info->flags =3D f; > > - info->data =3D n; > > -} > > - > > static inline bool cluster_is_free(struct swap_cluster_info *info) > > { > > return info->flags & CLUSTER_FLAG_FREE; > > } > > > > -static inline bool cluster_is_null(struct swap_cluster_info *info) > > -{ > > - return info->flags & CLUSTER_FLAG_NEXT_NULL; > > -} > > - > > -static inline void cluster_set_null(struct swap_cluster_info *info) > > +static inline unsigned int cluster_index(struct swap_info_struct *si, > > + struct swap_cluster_info *ci) > > { > > - info->flags =3D CLUSTER_FLAG_NEXT_NULL; > > - info->data =3D 0; > > + return ci - si->cluster_info; > > } > > > > static inline struct swap_cluster_info *lock_cluster(struct swap_info_= struct *si, > > @@ -394,65 +347,11 @@ static inline void unlock_cluster_or_swap_info(st= ruct swap_info_struct *si, > > spin_unlock(&si->lock); > > } > > > > -static inline bool cluster_list_empty(struct swap_cluster_list *list) > > -{ > > - return cluster_is_null(&list->head); > > -} > > - > > -static inline unsigned int cluster_list_first(struct swap_cluster_list= *list) > > -{ > > - return cluster_next(&list->head); > > -} > > - > > -static void cluster_list_init(struct swap_cluster_list *list) > > -{ > > - cluster_set_null(&list->head); > > - cluster_set_null(&list->tail); > > -} > > - > > -static void cluster_list_add_tail(struct swap_cluster_list *list, > > - struct swap_cluster_info *ci, > > - unsigned int idx) > > -{ > > - if (cluster_list_empty(list)) { > > - cluster_set_next_flag(&list->head, idx, 0); > > - cluster_set_next_flag(&list->tail, idx, 0); > > - } else { > > - struct swap_cluster_info *ci_tail; > > - unsigned int tail =3D cluster_next(&list->tail); > > - > > - /* > > - * Nested cluster lock, but both cluster locks are > > - * only acquired when we held swap_info_struct->lock > > - */ > > - ci_tail =3D ci + tail; > > - spin_lock_nested(&ci_tail->lock, SINGLE_DEPTH_NESTING); > > - cluster_set_next(ci_tail, idx); > > - spin_unlock(&ci_tail->lock); > > - cluster_set_next_flag(&list->tail, idx, 0); > > - } > > -} > > - > > -static unsigned int cluster_list_del_first(struct swap_cluster_list *l= ist, > > - struct swap_cluster_info *ci) > > -{ > > - unsigned int idx; > > - > > - idx =3D cluster_next(&list->head); > > - if (cluster_next(&list->tail) =3D=3D idx) { > > - cluster_set_null(&list->head); > > - cluster_set_null(&list->tail); > > - } else > > - cluster_set_next_flag(&list->head, > > - cluster_next(&ci[idx]), 0); > > - > > - return idx; > > -} > > - > > /* Add a cluster to discard list and schedule it to do discard */ > > static void swap_cluster_schedule_discard(struct swap_info_struct *si, > > - unsigned int idx) > > + struct swap_cluster_info *ci) > > { > > + unsigned int idx =3D cluster_index(si, ci); > > /* > > * If scan_swap_map_slots() can't find a free cluster, it will ch= eck > > * si->swap_map directly. To make sure the discarding cluster isn= 't > > @@ -462,17 +361,14 @@ static void swap_cluster_schedule_discard(struct = swap_info_struct *si, > > memset(si->swap_map + idx * SWAPFILE_CLUSTER, > > SWAP_MAP_BAD, SWAPFILE_CLUSTER); > > > > - cluster_list_add_tail(&si->discard_clusters, si->cluster_info, id= x); > > - > > + list_add_tail(&ci->list, &si->discard_clusters); > > schedule_work(&si->discard_work); > > } > > > > -static void __free_cluster(struct swap_info_struct *si, unsigned long = idx) > > +static void __free_cluster(struct swap_info_struct *si, struct swap_cl= uster_info *ci) > > { > > - struct swap_cluster_info *ci =3D si->cluster_info; > > - > > - cluster_set_flag(ci + idx, CLUSTER_FLAG_FREE); > > - cluster_list_add_tail(&si->free_clusters, ci, idx); > > + ci->flags =3D CLUSTER_FLAG_FREE; > > + list_add_tail(&ci->list, &si->free_clusters); > > } > > > > /* > > @@ -481,24 +377,25 @@ static void __free_cluster(struct swap_info_struc= t *si, unsigned long idx) > > */ > > static void swap_do_scheduled_discard(struct swap_info_struct *si) > > { > > - struct swap_cluster_info *info, *ci; > > + struct swap_cluster_info *ci; > > unsigned int idx; > > > > - info =3D si->cluster_info; > > - > > - while (!cluster_list_empty(&si->discard_clusters)) { > > - idx =3D cluster_list_del_first(&si->discard_clusters, inf= o); > > + while (!list_empty(&si->discard_clusters)) { > > + ci =3D list_first_entry(&si->discard_clusters, struct swa= p_cluster_info, list); > > + list_del(&ci->list); > > + idx =3D cluster_index(si, ci); > > spin_unlock(&si->lock); > > > > discard_swap_cluster(si, idx * SWAPFILE_CLUSTER, > > SWAPFILE_CLUSTER); > > > > spin_lock(&si->lock); > > - ci =3D lock_cluster(si, idx * SWAPFILE_CLUSTER); > > - __free_cluster(si, idx); > > + > > + spin_lock(&ci->lock); > > + __free_cluster(si, ci); > > memset(si->swap_map + idx * SWAPFILE_CLUSTER, > > 0, SWAPFILE_CLUSTER); > > - unlock_cluster(ci); > > + spin_unlock(&ci->lock); > > } > > } > > > > @@ -521,20 +418,20 @@ static void swap_users_ref_free(struct percpu_ref= *ref) > > complete(&si->comp); > > } > > > > -static void alloc_cluster(struct swap_info_struct *si, unsigned long i= dx) > > +static struct swap_cluster_info *alloc_cluster(struct swap_info_struct= *si, unsigned long idx) > > { > > - struct swap_cluster_info *ci =3D si->cluster_info; > > + struct swap_cluster_info *ci =3D list_first_entry(&si->free_clust= ers, struct swap_cluster_info, list); > > > > - VM_BUG_ON(cluster_list_first(&si->free_clusters) !=3D idx); > > - cluster_list_del_first(&si->free_clusters, ci); > > - cluster_set_count_flag(ci + idx, 0, 0); > > + VM_BUG_ON(cluster_index(si, ci) !=3D idx); > > + list_del(&ci->list); > > + ci->count =3D 0; > > + ci->flags =3D 0; > > + return ci; > > } > > > > -static void free_cluster(struct swap_info_struct *si, unsigned long id= x) > > +static void free_cluster(struct swap_info_struct *si, struct swap_clus= ter_info *ci) > > { > > - struct swap_cluster_info *ci =3D si->cluster_info + idx; > > - > > - VM_BUG_ON(cluster_count(ci) !=3D 0); > > + VM_BUG_ON(ci->count !=3D 0); > > /* > > * If the swap is discardable, prepare discard the cluster > > * instead of free it immediately. The cluster will be freed > > @@ -542,11 +439,11 @@ static void free_cluster(struct swap_info_struct = *si, unsigned long idx) > > */ > > if ((si->flags & (SWP_WRITEOK | SWP_PAGE_DISCARD)) =3D=3D > > (SWP_WRITEOK | SWP_PAGE_DISCARD)) { > > - swap_cluster_schedule_discard(si, idx); > > + swap_cluster_schedule_discard(si, ci); > > return; > > } > > > > - __free_cluster(si, idx); > > + __free_cluster(si, ci); > > } > > > > /* > > @@ -559,15 +456,15 @@ static void add_cluster_info_page(struct swap_inf= o_struct *p, > > unsigned long count) > > { > > unsigned long idx =3D page_nr / SWAPFILE_CLUSTER; > > + struct swap_cluster_info *ci =3D cluster_info + idx; > > > > if (!cluster_info) > > return; > > - if (cluster_is_free(&cluster_info[idx])) > > + if (cluster_is_free(ci)) > > alloc_cluster(p, idx); > > > > - VM_BUG_ON(cluster_count(&cluster_info[idx]) + count > SWAPFILE_CL= USTER); > > - cluster_set_count(&cluster_info[idx], > > - cluster_count(&cluster_info[idx]) + count); > > + VM_BUG_ON(ci->count + count > SWAPFILE_CLUSTER); > > + ci->count +=3D count; > > } > > > > /* > > @@ -581,24 +478,20 @@ static void inc_cluster_info_page(struct swap_inf= o_struct *p, > > } > > > > /* > > - * The cluster corresponding to page_nr decreases one usage. If the us= age > > - * counter becomes 0, which means no page in the cluster is in using, = we can > > - * optionally discard the cluster and add it to free cluster list. > > + * The cluster ci decreases one usage. If the usage counter becomes 0, > > + * which means no page in the cluster is in using, we can optionally d= iscard > > + * the cluster and add it to free cluster list. > > */ > > -static void dec_cluster_info_page(struct swap_info_struct *p, > > - struct swap_cluster_info *cluster_info, unsigned long page_nr) > > +static void dec_cluster_info_page(struct swap_info_struct *p, struct s= wap_cluster_info *ci) > > { > > - unsigned long idx =3D page_nr / SWAPFILE_CLUSTER; > > - > > - if (!cluster_info) > > + if (!p->cluster_info) > > return; > > > > - VM_BUG_ON(cluster_count(&cluster_info[idx]) =3D=3D 0); > > - cluster_set_count(&cluster_info[idx], > > - cluster_count(&cluster_info[idx]) - 1); > > + VM_BUG_ON(ci->count =3D=3D 0); > > + ci->count--; > > > > - if (cluster_count(&cluster_info[idx]) =3D=3D 0) > > - free_cluster(p, idx); > > + if (!ci->count) > > + free_cluster(p, ci); > > } > > > > /* > > @@ -611,10 +504,10 @@ scan_swap_map_ssd_cluster_conflict(struct swap_in= fo_struct *si, > > { > > struct percpu_cluster *percpu_cluster; > > bool conflict; > > - > > + struct swap_cluster_info *first =3D list_first_entry(&si->free_cl= usters, struct swap_cluster_info, list); > > offset /=3D SWAPFILE_CLUSTER; Ah, here it changes the meaning of the offset variable. The offset holds a cluster index rather than offset. That is why I miss it. > > - conflict =3D !cluster_list_empty(&si->free_clusters) && > > - offset !=3D cluster_list_first(&si->free_clusters) && > > + conflict =3D !list_empty(&si->free_clusters) && > > + offset !=3D first - si->cluster_info && > > offset !=3D cluster_index(si, first) ? Done. Chris