From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66492C001DF for ; Tue, 15 Aug 2023 22:30:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0FBE8D0018; Tue, 15 Aug 2023 18:30:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A997D8D0001; Tue, 15 Aug 2023 18:30:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 913018D0018; Tue, 15 Aug 2023 18:30:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7817A8D0001 for ; Tue, 15 Aug 2023 18:30:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 421FBC0DD1 for ; Tue, 15 Aug 2023 22:30:35 +0000 (UTC) X-FDA: 81127784430.30.ACC74B1 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by imf22.hostedemail.com (Postfix) with ESMTP id 75F87C0006 for ; Tue, 15 Aug 2023 22:30:33 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=fuO4sjjh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692138633; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7d1/EJSahT73cr9ZlY19BewTHK2xn5aHPxZEg9xbznE=; b=zE+pER+NHsXeFRuS5vO/FuZ4WJWxsfYau+lhi+lEYoA0fWWP1EORP9tIyVztiU3yGBGL3l fEski0QBe/SufxT680AU8BEuVwZd1m3oyEBE+cRPAShUukY6Ml4f2J0m4HeXSO8jpyuWhn 2sJ/wkT97YlkeHoNnCWw+zkiRzFvvXI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=fuO4sjjh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692138633; a=rsa-sha256; cv=none; b=TjFWckM8ORlO4v2LWLvD1EZADSXUNWmy7iY2qhiI/B4usd32Sv5eocsfwz/HGf8Zpw+T9o hJwabK+mgD6wjWBqTFi0yBdUrj2+h1zuBZZ395fzkqbcxstI9g+twDiWou/m2dN62oc9jR GHUfBbgOaZwspEG1i22IB6dM6DpFeTM= Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-99bfcf4c814so783282566b.0 for ; Tue, 15 Aug 2023 15:30:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692138632; x=1692743432; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7d1/EJSahT73cr9ZlY19BewTHK2xn5aHPxZEg9xbznE=; b=fuO4sjjhidMzbvLnT9dUoAAFNvsL80E86QRBRvCG68qw2PwOECF7uNFobvdM6y5lKU qjcVVEaRDyXoNJXCYtMbXDuFoW8yIJgZ/4A4kWYjFT6JFliWW+92D+HIU2wNtgHwNedU 1xCbPa5QAEwjL7mJS57hZdlqigBCHDZgbU22cr8HT5A9TWDl8vuoeELdG11BCoLOmsW0 aqkFl6+maxXHPmNVNM5TO5q2RqJEQMl0OBrEyb5q5roH6f51gfGw738qcgaByXAb6DYl /roNcORi2zIPZonlrN42mv83ldo3eF5wFAcR9jh7FLnQ4+Zwun8L3iXx6Eug/dsj6TVu C0Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692138632; x=1692743432; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7d1/EJSahT73cr9ZlY19BewTHK2xn5aHPxZEg9xbznE=; b=ZVfMtxnYU4npp39XYknshILhiEEGt1m2lML1dGjp3ie6x5Q0xAdHuqNQfwF12PsDcd 7iienbcfrS06qXbVCji2ZfZdjFvN+kHKCYE0DIpUs901EqkQbjtluqgcJktSoJ8fUH4a 5vveky2akQAY8tjwOVXMAhlPIqF9WMZ01A3VB3dt9Dg32r7TDDppNrDgOZbsds7eq2Gx ZKfsXPOx5FXZ5WPOxjsuMtlU2kNlhRQFcADnN7nUZKXFuXoFyyRj29p0mKkkxFtXefuj C87bc3tTDqsIFUIGNGdbICgPwAvqEoFIr9HYGwtAdjq0Ukvmh384ATvrNsaDTDdr4ki4 am3Q== X-Gm-Message-State: AOJu0YyVpl7MthYckJTTcfZ9QdPkoaJfpR0g8qAlFa5uhEaxKpwfwwOR A3OFAkUK4F425LMBCQE7uIH0uDABIqECHaSj7aEScA== X-Google-Smtp-Source: AGHT+IFefgKJ3rlJrwcdcShWAye+TdzZGubEtXnhS2+tj43kxBszt5uVbjyXVBBxySosNukp2MmnF7gRjb0d1rz7LGI= X-Received: by 2002:a17:906:9be9:b0:993:dd1d:8251 with SMTP id de41-20020a1709069be900b00993dd1d8251mr31205ejc.28.1692138631681; Tue, 15 Aug 2023 15:30:31 -0700 (PDT) MIME-Version: 1.0 References: <20230620194644.3142384-1-yosryahmed@google.com> <20230811141920.2f9bab6f21ab635ff1bd63ac@linux-foundation.org> In-Reply-To: From: Yosry Ahmed Date: Tue, 15 Aug 2023 15:29:55 -0700 Message-ID: Subject: Re: [PATCH v3] mm: zswap: multiple zpools support To: Chris Li Cc: Andrew Morton , Nhat Pham , Konrad Rzeszutek Wilk , Seth Jennings , Dan Streetman , Vitaly Wool , Johannes Weiner , Domenico Cerasuolo , Yu Zhao , linux-mm@kvack.org, LKML Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 75F87C0006 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: wijekp746iejgdekwxjmp4iigs4aqsp1 X-HE-Tag: 1692138633-887531 X-HE-Meta: U2FsdGVkX18MVZ2raA8d0eNjMQsOIBxZLYdgWRz88Sj0N3kffwgD3wqBri3eX38boVyehvrr7LcIUBk/zN38NenmRB5en68cPX46T1dQvtkGSUDGGc33IuTb1+b4paEmmhS8PUr4s2e1LLbL42rtEsklEVI6MKidit8wj7bDoVGjr4qc+Hf6PrmfU1V6SWEsWZhMe/l1QEqneGA+Pk4wZCU3cd4CbWH1l2/9pqLxlVJPYmfyIm3/exH6e7ObpLpQqUbx/s0yFP53wlqWcJUw+2tt+E3KmiGfR+yNzIzAGOzV/YOkLXliBitfCjxgiqWVFxNOwlPMenTgYH+9tmHPVBVUnCwOa4bxObhw44x+Q2PTnCclsqU2+H7KPsqm6vw5QdnzYyvUG+VuDuXCz9meh6qLjk5chn8I83zGx8tIZEb0jXM8jrzBUZTWN5I1Ir0cTAfhBNGOs9AiZJFo4Q5HiPyr87uwypj/lUiwEh4ok5xbvFm5R3wwQEtX/YIIIH4/M69Z9kBPH6qRZdP4hkXmjUhjBh49ixQmB+KG07MPXj6Tph02UBAXqiC+iG3gehb6RyzBeHIPX8YpXR6hDIV5s++blYlKiC7Y/msIqqoMIJXNIYXMxHuulGFiw7FxGTDhjZEBFF0lG8y3cyMp24o2LGCeuqtNwIETXUTE+0T/WnjpDtD+V5IV3CQAW+3pEOFPaOr2vtaGpFxQB1udX8codVJ7iwg8gLY68q/cMdtcvuH+lO/IBskeh69NixGn5nkceltGLXEWK5lDwGePJZbphQr1Xr1xmZnnqxrI1OLii3AnbGF0VT3dKNNOirrlNksC8sENDyiLLltAd/3ngvDGtoVW5mO0lfbIZigGDfoB8reD6+307bm2F9PP9cfj7kmJjTsh/mpvSLtLg4iiWGrW+8xcXkSrQYJE/Y9Lp5tzVqG54pXngbsq02DWD++SiTJ8DeNvimod1LKGgatTUQ0 SDOcI5ya w1sR0wks2Cc8+/BMAS9t0fxN9x7Y6GAN4elDnUUFL6qron4Y5kQWx/hu2EuPmv11SWTP+inCwhVEvRFZtPqtExNt93CBnhzCBSTAlgVAujC+vM3fjls2MIjhbh6sJP1bsb/zGUEjSIkwu8+D/prCUf+3D3LMqLnaRvMQ8HrLzJiICo34vIS0ck9TOv5wwBdVeEAEpc8OcF3NxnsWYj346uqVz4Sly4DkIKQ/r1NlKP1lwQLZ4HbXLhcK35Ix+ObQA1GXelbHWbrIavSdCLg/zMcOoOskbM0XtGnJo73v6meJoTD0sbmjJxpbyJY0FyJZaBz88UJlVWbKA9EJNBvqYsw9QKoIhTvDwFBD0pN3V62/Qr4UgA+JoKAkmqA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 15, 2023 at 3:22=E2=80=AFPM Chris Li wrote: > > Hi Yosry, > > On Fri, Aug 11, 2023 at 4:21=E2=80=AFPM Yosry Ahmed wrote: > > > > On Fri, Aug 11, 2023 at 2:19=E2=80=AFPM Andrew Morton wrote: > > > > > > On Thu, 13 Jul 2023 03:35:25 -0700 Yosry Ahmed wrote: > > > > > > > > > > > > > I'm experimenting with some other zswap changes - if I have > > > > > extra cycles and resources I'll try to apply this patch and see h= ow the > > > > > numbers play out. > > > > > > > > That would be amazing. Looking forward to any numbers you can dig := ) > > > > > > So this patch seems stuck. I can keep it in mm.git until the fog > > > clears, but would prefer not to. Can we please revisit and decide on= a > > > way forward? > > > > Johannes did not like a config option so I proposed it here as a > > constant (like SWAP_CLUSTER_MAX and others we have). This is a value > > that we have been using in our data centers for almost a decade, so it > > I dug up the previous V1 discussion and this V3 discussion thread. > It seems obvious having multiple pools having locking contention advantag= e. > The number does not lie. > > However the number of pools is hard to decide at compile time. > > Regarding the per CPU pool. That might work well for a small number of CP= Us. > When the system has many CPUs e.g. a few hundreds of CPUs. It means havin= g > hundreds of pools which is a bad idea. > > How about just setting it as a run time value(size/bits) and can only > change pool > (size/bits) when zswap does not have any active stores. I was hoping we can add the basic support here for multiple zpools, and then later, if needed, extend to support runtime dynamic tuning. Adding this will introduce more complexity as we will need to lock all trees and make sure there is no activity and alloc/free zpools. If a limitation for compile-time constant is observed we can do that, otherwise let's keep it simple and incremental for now. FWIW, we have been running with 32 zpools in Google's fleet for ~a decade now and it seems to work well for various workloads and machine configurations. > > Chris > > > has seen a ton of testing. I was hoping Johannes would get time to > > take a look, or Nhat would get time to test it out, but neither of > > these things happen. > > > > I obviously want it to be merged, but hopefully someone will chime in h= ere :) > >