From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99CE42772D for ; Tue, 23 Jun 2026 11:37:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782214670; cv=none; b=KGTIlBWh/xfGGbo//ATDlPEc5zv4MQIVfBT8uaX9g+2FUzJQT5W9/6IaUOhYyofDXtjxzupZ84TulsS2C0STwOb4d+ek6lDyf2rzFLRYgC9LPnV612ARN80N8x7qykKh043uTD1paaQ3/ufWO1ZwMlKtPL4MpZWb0BIJG/BGWsI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782214670; c=relaxed/simple; bh=t/a2x5Ghb20XmVHJR+ek7GnEWJsjkaghf0C6SjAspjc=; h=From:To:Cc:Subject:In-Reply-To:Date:Message-ID:References: MIME-version:Content-type; b=PNYKCWqYXBImjKMFO3gK2TedTYvV4xTzXKFEBYJ20WQCyb9ZAgS7N88R0RosKfk0svWCGqhbNBTArQrkCHvFCb/MPB2vf2uKWBnyOZYSMLK1fbG9a/7o9mGquJfjh+3h6P++fKUjYiCfcYBh8E/I+uDhphD7Gkf6Uwo3oy1dt9Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Sc5QhCyR; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Sc5QhCyR" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-c85b73ffb52so2380457a12.3 for ; Tue, 23 Jun 2026 04:37:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782214669; x=1782819469; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:message-id:date :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=t/a2x5Ghb20XmVHJR+ek7GnEWJsjkaghf0C6SjAspjc=; b=Sc5QhCyRA8DV1FEM4g/ccG5iMVPoEGXCC1Z2+On2noqtkkCrn0fXdi8SE44kWG8VrB CQtoctlLbtaLCLSHiol5UM/ygyUEdnv8ARMy7S9XJi9y0ffTBfq5DsloDspoB2h28bTZ WDvHe3UIrf0lOcX+jNqiOlLEbi5L6Xyhy4w478iuzCyPL3HCphv1+Xt77omAJdw9qjN4 +qYWLAJjIN5m+td0nI2A9fH9Q9oqEikQiuYR1V7teasStF6nsTpIMBFNX5pUYcYl9GOA ML96FXkpco+wbuoEv2cBTuS4wh+i4U0HRcVfmE7RRDkSF+nlJ/BVmuP71ZooHY80zjPV CuXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782214669; x=1782819469; h=content-transfer-encoding:mime-version:references:message-id:date :in-reply-to:subject:cc:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=t/a2x5Ghb20XmVHJR+ek7GnEWJsjkaghf0C6SjAspjc=; b=NyAhh6J/KlaWkCtYQdzcRvYTtxtTx62rIOs5t04AAjy41RTxrVKJKnvOzs0Xr0cBf7 YDX20Cw4yJhpicY9qrbEzH0jbIcd5nv39XeI5SFtejYPuchclPYOLCvcfY+o8hoE06Jm 0Z3P+nEpd9Pd0hoEvb0ZTizR96vnI+umX87jxwv0WXuox8VtpvHUFEsuwonneipSsXka 0gRy6eUT0E6Qkr/zxTIIXQlPbfXbE3Yu5T3+dfizE0poV90ellS/Na3q+5KPRYfECgGP bLVF3YgrgmkdTDgkgV7QGJiWRPM7sRyjYawgmqhj7+/g/dgxJFpWOS1Sg+szY4bSyfbm Xtgw== X-Forwarded-Encrypted: i=1; AFNElJ83PRTu3NIfxWK1diVSlfIoYPSwXywPgndlzKAHdNgbalqsfp53TfJ/rduoiYkqRh//GC05kXqBzLuysaA=@vger.kernel.org X-Gm-Message-State: AOJu0YwmYTP9jH1+sbI669PN1BDSjgIyJaZnWCWpm6EZWs1L4D6e7AJm vCZwFt1C9d5bBUXdrBfvfF2pu9b1grV3+zo/lQPk0yrWewlmE4EN1YJl X-Gm-Gg: AfdE7cnUbM33Ofp56/k9+1XH5gwVJOTDHxhZZxpiQNWFIEqwF25TakdYDod8T2ZrNfy Ua+PTajrseoGyFQvFrcXvAv2AXW4Yirnm3RBId4w4Ulr76nXGCuJqm/ut1Mqr3WPeOJPNK+5mL9 ngfyJOU2qHqkvqx0TicppWoOsrOrwDDorcPyTSHXT5i+OYgSOTI/Jg2mrWNwfgL9dJ5uBq82rKv zc4zN8Pno7JRB8+GwHlp/s/n5dorVz+EsKmDWNAfw/5H7BZX9R17znobjABGXW0LsZ9lzCVZOr+ Zv1HihAHP0q/KCTXIG4r17yzWvQKXFGDcKtXhM4y7U1CxWsXq2i3IbAowz8K0BUU7A+UYXqmdy0 sqXoAGrkUPggjml33GjOVU6akDZm89wonc5hFeYui4ZRGqgHIOYSA3iDN8FNTYVBAGOD7WNuuvk n/J22TrJgUAZooFzI= X-Received: by 2002:a05:6a00:4b51:b0:842:4327:6cfb with SMTP id d2e1a72fcca58-84597123e04mr2464203b3a.46.1782214668486; Tue, 23 Jun 2026 04:37:48 -0700 (PDT) Received: from pve-server ([49.205.216.49]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84564ef4c09sm11953864b3a.61.2026.06.23.04.37.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jun 2026 04:37:47 -0700 (PDT) From: Ritesh Harjani (IBM) To: Barry Song Cc: linux-mm@kvack.org, Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Youngjun Park , David Hildenbrand , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Sayali Patil Subject: Re: [PATCH v4 2/3] mm, swap: allow archs to override SWAP_NR_ORDERS via ARCH_MAX_PMD_ORDER In-Reply-To: Date: Tue, 23 Jun 2026 15:02:07 +0530 Message-ID: References: <33ydyd82.ritesh.list@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Barry Song writes: > On Tue, Jun 23, 2026 at 3:05 PM Ritesh Harjani wrote: >> >> Barry Song writes: >> >> > On Fri, Jun 19, 2026 at 12:41 PM Ritesh Harjani (IBM) >> > wrote: >> >> >> >> SWAP_NR_ORDERS sizes a few small bounded arrays inside THP swap >> >> allocator code (nofull/frag cluster lists, percpu_swap_cluster's >> >> si/offset arrays, next array for rotational device). This currently >> >> expands to PMD_ORDER+1, which only works when PMD_ORDER is a compile >> >> time constant. >> >> >> >> However on architecture like PowerPC Book3S64, PMD_ORDER is a runtime >> >> variable which depends upon which MMU is selected (Radix / Hash), so in >> >> that case, PMD_ORDER cannot be used to size the static arrays. >> >> >> >> This patch provides an optional ARCH_MAX_PMD_ORDER (upper-bound) >> >> override for such architectures. The memory overhead on enabling this >> >> override is negligible. Even if we make SWAP_NR_ORDERS runtime alloc, >> >> default slab padding could cause some memory waste. Also we lose the >> >> per-cpu cacheline benefits (for percpu_swap_cluster) because it might >> >> cost an extra cacheline indirection overhead in swap_alloc_fast() for >> >> fetching si[order]/offset[order]. Note that a fully runtime >> >> SWAP_NR_ORDERS was considered in previous version but was dropped for >> >> this reason [1] >> > >> > Do we know the maximum PMD size? >> >> ARCH_MAX_PMD_ORDER will be 8 on PowerPC book3s64 with 64K pagesize. >> PowerPC Hash MMU with 64K default pagesize supports PMD size of 16MB. >> >> > On arm64 with a 64 KB base page, >> > a PMD can be as large as 512 MB: >> > https://docs.kernel.org/arch/arm64/hugetlbpage.html >> > >> > One concern we have is that performing I/O on such a large folio could >> > incur significant latency before reclaiming any memory. For this >> > reason, on arm64 we initially enabled THP_SWAPOUT only for 4 KB base >> > pages: >> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d0637c505f >> > >> >> That's not the case on PowerPC. Max PMD size for Hash will be 16MB. > > Yep. A 16 MB folio might be fine, although I'm not sure whether > splitting a 16 MB folio into eight 2 MB folios would help much. > > For 512 MB PMD-sized pages on arm64, one possible approach might be to > split them into 256 × 2 MB folios rather than all the way down to 4 KB > pages. That could provide a better balance between I/O latency and swap > performance. > Fair enough. I guess this can be looked upon but is outside of the scope of this work. For now Radix with 2MB PMD is the default on latest PowerPC, so this will be slightly lower priority for me right now. >> Also we still need this patch since we can at runtime choose Hash or >> Radix MMU. So, the main problem this patch is trying to solve on PowerPC >> Book3s64 is enabling this feature w/o impacting any other architecture. >> W/O this patch series, we can't enable it, since it gives build errors. > > I see. If possible, please mention in the changelog that the maximum > PMD size on your platform is 16 MB. Sure. I can do that. > In that case, the I/O latency concerns I raised may not really apply. > > w/ that, please free feel to add: > > Reviewed-by: Barry Song Thanks again for the reviewing this patch series. I will re-spin the updated version (with additional details in the commit msg as you requested) in a couple of days. -ritesh