From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 459E7C00144 for ; Tue, 2 Aug 2022 02:50:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C815E6B0071; Mon, 1 Aug 2022 22:50:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2FD16B0072; Mon, 1 Aug 2022 22:50:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAA558E0001; Mon, 1 Aug 2022 22:50:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9D7DC6B0071 for ; Mon, 1 Aug 2022 22:50:16 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 525141A0C69 for ; Tue, 2 Aug 2022 02:50:16 +0000 (UTC) X-FDA: 79753123632.22.E3EC252 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf08.hostedemail.com (Postfix) with ESMTP id 8435A1600F8 for ; Tue, 2 Aug 2022 02:50:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659408615; x=1690944615; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=Tmxpd8lSQ+gGqxV2DLu70VQAoTc9jXD0B/zGr+ox7gU=; b=bGfzZTaPajrKwBnmpyiv4JXEfIQL+n+wXJ98V3vR4HwAfau81gOWgeis le+lTIpkuMMSGTSAD77pNVC5ifTqPSiKBC7bvciBO9YBUurokBjbX62vi jE7e1ytb0Zmvqm9Ks6l/VRdQwnYuFKuEIAupAwShrL+ZKKRYwKeYPl35d hEhCH5801VQ1Rxn2XSskf+3gs5xj3KOA1LrL3vNWJtvBJpqDBsrxBUQsR GAWqeQZHUnfscS78IMCUmIPO+t29wcabG31f4q3BMRaoHnT7DAdEUNsw4 s0oY5CflVGMFTHjyK6XJp1FpZI8G3axM3UDA4uhDf+/gS0XYY0u436lp5 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10426"; a="290518415" X-IronPort-AV: E=Sophos;i="5.93,209,1654585200"; d="scan'208";a="290518415" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 19:50:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,209,1654585200"; d="scan'208";a="630521764" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga008.jf.intel.com with ESMTP; 01 Aug 2022 19:50:13 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Mon, 1 Aug 2022 19:50:12 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Mon, 1 Aug 2022 19:50:12 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28 via Frontend Transport; Mon, 1 Aug 2022 19:50:12 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.107) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.28; Mon, 1 Aug 2022 19:50:12 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yw95UuLukKRGyx9UjxVJZCtINS6bzfdrQwU2jeW6j9f/fUAFvfMyLkjHXJN8RNONn1v7uxJt+0ikR9byYYP7MbTwi38aAqSv4Q1ePrsGq6JPHsKVLxaYdfoJuHuyVCbgoiVGyLSB3HwIt6bZW+jr0G2lmApBgohNGqrRJpFSFZCUqz1estX5m00bQcTZuXaTSjQ5PdI3zOiErSYZuRAzDCbtZuBGvKJpIuIsvQiMtS+7nH36d/7ABVstDZXwK36DxcRUWIAz0YrQ4moKXO4allSPMpnw3XDlqttH3ufMUKoMDFZyJjAJO5Hd1lmNnWfOXODjebEPrLueqMbNR+8yvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hQTTKPGLIuL2pif6D2xy4WtnlDsFFx70oWcJg1pJfx0=; b=Iyw0BFFy6eKEPAJvknspvmH2l2pS25zLBY0zcI68tGlqsD4AC0o0MnEaoJNkoN8zSjcjny7rid2Sd3W1iIQCkeJsFtXKGvCP8j+QlQtrYtZ7cqn8F5vmJVBbf5gGBge6L2Siu1zij5YSyzIBy1JCB0lT7qt+LfucETSsmRu50+1t8qim/XWGtNJLkyhmzKNw42M2VQzea8zCl86/LbLz9jKTPVZOiHoVsIYG2qoqQQ9o3T/PX4YxnESBBDaufN4dZCr30uvFeNL0PpIFEnqCGpCUYMcwoJ81UjhDfb7KXgEaoTgNrgTVFxBG4Kbo9IbvMTDAOuZ2pgFDIOncHMpVxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MWHPR1101MB2126.namprd11.prod.outlook.com (2603:10b6:301:50::20) by MWHPR11MB0047.namprd11.prod.outlook.com (2603:10b6:301:68::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5482.11; Tue, 2 Aug 2022 02:50:05 +0000 Received: from MWHPR1101MB2126.namprd11.prod.outlook.com ([fe80::9847:345e:4c5b:ca12]) by MWHPR1101MB2126.namprd11.prod.outlook.com ([fe80::9847:345e:4c5b:ca12%6]) with mapi id 15.20.5482.016; Tue, 2 Aug 2022 02:50:05 +0000 Date: Mon, 1 Aug 2022 19:50:02 -0700 From: Dan Williams To: Aneesh Kumar K.V , , CC: Wei Xu , Huang Ying , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , "Linux Kernel Mailing List" , Hesham Almatary , Dave Hansen , "Jonathan Cameron" , Alistair Popple , Dan Williams , Johannes Weiner , , Aneesh Kumar K.V , Jagdish Gediya Subject: RE: [PATCH v11 1/8] mm/demotion: Add support for explicit memory tiers Message-ID: <62e890da7f784_577a029473@dwillia2-xfh.jf.intel.com.notmuch> References: <20220728190436.858458-1-aneesh.kumar@linux.ibm.com> <20220728190436.858458-2-aneesh.kumar@linux.ibm.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20220728190436.858458-2-aneesh.kumar@linux.ibm.com> X-ClientProxiedBy: MW4P223CA0002.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::7) To MWHPR1101MB2126.namprd11.prod.outlook.com (2603:10b6:301:50::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 005912ae-ac97-40fe-33b5-08da7431b3f2 X-MS-TrafficTypeDiagnostic: MWHPR11MB0047:EE_ X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AlrlYmf1mun8KJi0PYYVFVt6QExSPJYZgC33cvAe39CiQDyEb/1Ty3fUFZJ2E3wv9lgqphXtBOTuOk852rMLG4HNoAfPDMfB0RG3vI904sy4dRiAslEnVHHGjvVD4j0Qdb/jSlfYNhgDSYKmlgfqfj3MADdlfHxyVpmNCZ97LD+a9d1FAr8uFUQMSQu3XAogF69LllGtXmIcwZZL2ExhPN5JY2kWO/8NvHdCIwRR3pFuEhUWA8t/GFq32lM1up7DTHYm/g6nEy7nPq+MHJiCi4dwDbjgjK6hGP23oaQkzHEW53beOf06RaBQDLsAZbLGiO5cixYFpvHlFVq3ifYI+djoQS5G0nzQUonMztlXs8SMijJrHnxuVaQkwC6XUZidDTCNC051yYQ9SVBlRCDv7KPVznlxAAQ2O8ZfXTWOcEPGrZi/sQ0hLVcn2aWYF4vNxElZVmJCR/XJWdFFqSvobTr6zs9R+qsxg3zd3jrIZw2ZV2rOPl4/HHUKKQvcKfteWH/13q20H/w/24mZFNADkQ/NUmwDz+hQdo8w2OzGbzzlDm6URn0dKFXlBwCflmJ8oSLqQVQVaR1asVnPYwiA37Y+R9tneBkcSIAYKQOEZLq5AK/iNTnVbLQAeYxXNFxBvSgw30rxkI5WdAsWwGCMpxn0bTSanWRyUprurls+xDuJs1Tda96Q2CBrZlN9PRwgewFlHk6IrjNiLhp5ZDdNh3bTxap8tZtauKtgkZ+CK9tPEGYeFbkcINQ+n3klZpJfn1h6FBA8/4HWHoLTEw559iX1CEUBAENPtmviL1cLkKJD8kEn7ycpKigy7ktPGdfF X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR1101MB2126.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(376002)(346002)(39860400002)(136003)(396003)(366004)(6666004)(9686003)(41300700001)(38100700002)(6506007)(186003)(83380400001)(7416002)(2906002)(6486002)(966005)(26005)(6512007)(478600001)(316002)(4326008)(66556008)(8676002)(66476007)(8936002)(54906003)(82960400001)(5660300002)(66946007)(86362001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2FwvC6Jzwh0czd2luVosdOVhlPuOlt7RLChpjh7Xa94Zr2TPF1qApiw2u2l2?= =?us-ascii?Q?/LV0n1JGm8RMrL27E7jdh8dyIVbVubKZw73tJl6Fleqw58w4OQ+EvOqQ2bhf?= =?us-ascii?Q?Q3bN0Y9jKEeV0r4A86BW9TbHkIydLenfPtens023vPD4hIvOVCN7bnpS86E+?= =?us-ascii?Q?9x+xG7vO73Y6G7tgDXRNg5t1ncKNfIC5qBVgXyO7gfC7g6k7ngInWnUkQ62d?= =?us-ascii?Q?ak6pgXkP9sOmwofOxxVJywO3dTexj3Cx6nk8ZxRouPH3JSq8y4neGQPMoPYx?= =?us-ascii?Q?UcIb/35bnmzplCvikZdZ4TXx3Iij0oAXXNl1fnMc82xxCZALEUoil9M4qACf?= =?us-ascii?Q?9eQBn8wbDomZSi4R1JynCfzhx/y+K8yRmhtY6lnB27suvj/vMjwRVlzCkhCm?= =?us-ascii?Q?2JVKLyVnf+9gooBcwTs6eRrwPTqLe5qXzJp9w2dZEW9TIwiK4wUDjpQsu2QX?= =?us-ascii?Q?+5QSvzpz0sojevbfdD/Dqg8NQCjiwvzuCGEcCMR/51ztbTbziHoNrLNHHAMe?= =?us-ascii?Q?s3zLGvkdAiQR3DZAn6fzgTmbva0zELXbtbwha64waYiJ1wJatGPEuneaP2u9?= =?us-ascii?Q?j21HFOoahZ4QL5OI+bAl8ebXr1i1pRn0xJx4cbDYJxYcXXwoe3kaYJYzq3Bq?= =?us-ascii?Q?VSc69jwSokz5QaPVaPljVBiP+/n8vrRlPmWD8MWQugdp2jNm6MWLyd+ru8PZ?= =?us-ascii?Q?v9tULKrXa2RdExgM8kIEP23fhtXcx1cxfHugDblT97at5+SLPFpbsJjBF87O?= =?us-ascii?Q?s9J2U+Xm/jNVBdNQFWAE3DEVw5mNJ/8kqKbXDSiolTOq3c+5F4+A3JCBu+aR?= =?us-ascii?Q?F1taCHHFZtG9zFGjxhwNcViKJn2MRR45A0/mzESq43BL3u/TgUw+65qwTf60?= =?us-ascii?Q?xwbE/m0tppXglXP9hvpZMDRWILVTrAFMVv5R3bFxK1AmTQ15c/1VUG/J33zL?= =?us-ascii?Q?gDYhjlhSI+Ma7LGA8bcHprFmo2Gd7miGncvWNMC77dw/7HW4QZ+m7apbEyd2?= =?us-ascii?Q?qbcVmyGK7sCgKELGdfTGj8jJM6s6RHDC5sM4GLfpy0113crZEXQu3fTLz6S0?= =?us-ascii?Q?pj2Rj52APwiPTripG6bLLCij6ejMhyc+7+y83G4OlkTReKxwJTphUHW9TQ3u?= =?us-ascii?Q?6dGqdSF87zjIufBh8MRVNEQpF3USv81pVMgxk+OXjlW8BL2AnFwLFkKyBORz?= =?us-ascii?Q?sUfvz5L1Zq2yXkCiS0Q++n5BhhmqnsW3ibG18Us3Rpd/MXc0OCZb1wTOxJMg?= =?us-ascii?Q?DJFGBiPeAqDYPcTWJZZWBESmp1ZPcc7VPi/THEzhDkFM1EOm1BYwpBNhI9cr?= =?us-ascii?Q?5uv/+jeItKnbIBMLB11kkd7CnrzuvMW2J9OYJSzGAk7MNhGnDoaeO7D8jyBR?= =?us-ascii?Q?QFERUOU0QL8fr38p5Jlra4C0Sz789keIPhWY1g3+L1ycsFQX09fZNNrcHZZu?= =?us-ascii?Q?NYHqk3n/mF3dmKGJT+fS53F6wHOfcK8qpaox6NYpDwO9SyxS2fvC7O7VytRm?= =?us-ascii?Q?goAcQ0ZLpSsAOIH2vslbFxVmWojuo510nGnSF8j3jxeb4Ou8K/Rfe7aCJUuq?= =?us-ascii?Q?hZ206fxva4I8V67Be2EgSm2RaJ7P7hjO/7VZitaKHxEKBpynIJz6zCnhgJMy?= =?us-ascii?Q?dw=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 005912ae-ac97-40fe-33b5-08da7431b3f2 X-MS-Exchange-CrossTenant-AuthSource: MWHPR1101MB2126.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2022 02:50:05.0372 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UWaRsnaxf/sX5DtVPup3GZB12VJpO95LBIlQYu2GTAzAveZCYzq4NrBBo4evMssyJ8Xr1iPyFmtIVcnnOx0dKe68zLoO8/M6UPEaQ1E5AQs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB0047 X-OriginatorOrg: intel.com ARC-Authentication-Results: i=2; imf08.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=bGfzZTaP; spf=pass (imf08.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1659408615; a=rsa-sha256; cv=fail; b=CulxfQ8eKfvZAU0yv4pCuJRfNzkGHoK3UTUyNnnmKfR41U7Le4OHjTxf9e1isON6r5UnP1 v7x2E2HpeyPtS3ZXN208SfY6SlVgx2/vz555GcczTMwTIXd5O+vQwuhTUCAP6tK7PlbczW Jm/xXjDZ5PmBWRNvxAlA2T+uScS/zuc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659408615; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hQTTKPGLIuL2pif6D2xy4WtnlDsFFx70oWcJg1pJfx0=; b=udNDVlI19xwSTuOAJD01Yw5HA/szpRc09ABKCK8O1/NNahYeRocgYJD6HeMtgaT58H8DK0 P6hZ/iaJ9ldwfXRzltvjlxjVYZl3GP7Qdqwyr6JHJYCg1y5nlkn3k5HJ9uJLLkRZtagEkc ymTO234D6Du5ADg/pJSSt3UzV4Xxi9o= X-Stat-Signature: etokcuqtpb75mo5kspo4fdjc56wcm15t X-Rspamd-Queue-Id: 8435A1600F8 Authentication-Results: imf08.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=bGfzZTaP; spf=pass (imf08.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1659408615-757917 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Aneesh Kumar K.V wrote: > In the current kernel, memory tiers are defined implicitly via a demotion path > relationship between NUMA nodes, which is created during the kernel > initialization and updated when a NUMA node is hot-added or hot-removed. The > current implementation puts all nodes with CPU into the highest tier, and builds > the tier hierarchy tier-by-tier by establishing the per-node demotion targets > based on the distances between nodes. > > This current memory tier kernel implementation needs to be improved for several > important use cases, > > The current tier initialization code always initializes each memory-only NUMA > node into a lower tier. But a memory-only NUMA node may have a high performance > memory device (e.g. a DRAM-backed memory-only node on a virtual machine) that > should be put into a higher tier. > > The current tier hierarchy always puts CPU nodes into the top tier. But on a > system with HBM or GPU devices, the memory-only NUMA nodes mapping these devices > should be in the top tier, and DRAM nodes with CPUs are better to be placed into > the next lower tier. > > With current kernel higher tier node can only be demoted to nodes with shortest > distance on the next lower tier as defined by the demotion path, not any other > node from any lower tier. This strict, demotion order does not work in all use > cases (e.g. some use cases may want to allow cross-socket demotion to another > node in the same demotion tier as a fallback when the preferred demotion node is > out of space), This demotion order is also inconsistent with the page allocation > fallback order when all the nodes in a higher tier are out of space: The page > allocation can fall back to any node from any lower tier, whereas the demotion > order doesn't allow that. > > This patch series address the above by defining memory tiers explicitly. > > Linux kernel presents memory devices as NUMA nodes and each memory device is of > a specific type. The memory type of a device is represented by its abstract > distance. A memory tier corresponds to a range of abstract distance. This allows > for classifying memory devices with a specific performance range into a memory > tier. > > This patch configures the range/chunk size to be 128. The default DRAM > abstract distance is 512. We can have 4 memory tiers below the default DRAM > abstract distance which cover the range 0 - 127, 127 - 255, 256- 383, 384 - 511. > Slower memory devices like persistent memory will have abstract distance below > the default DRAM level and hence will be placed in these 4 lower tiers. > > A kernel parameter is provided to override the default memory tier. > > Link: https://lore.kernel.org/linux-mm/CAAPL-u9Wv+nH1VOZTj=9p9S70Y3Qz3+63EkqncRDdHfubsrjfw@mail.gmail.com > Link: https://lore.kernel.org/linux-mm/7b72ccf4-f4ae-cb4e-f411-74d055482026@linux.ibm.com > > Signed-off-by: Jagdish Gediya > Signed-off-by: Aneesh Kumar K.V > --- > include/linux/memory-tiers.h | 17 ++++++ > mm/Makefile | 1 + > mm/memory-tiers.c | 102 +++++++++++++++++++++++++++++++++++ > 3 files changed, 120 insertions(+) > create mode 100644 include/linux/memory-tiers.h > create mode 100644 mm/memory-tiers.c > > diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h > new file mode 100644 > index 000000000000..8d7884b7a3f0 > --- /dev/null > +++ b/include/linux/memory-tiers.h > @@ -0,0 +1,17 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _LINUX_MEMORY_TIERS_H > +#define _LINUX_MEMORY_TIERS_H > + > +/* > + * Each tier cover a abstrace distance chunk size of 128 > + */ > +#define MEMTIER_CHUNK_BITS 7 > +#define MEMTIER_CHUNK_SIZE (1 << MEMTIER_CHUNK_BITS) > +/* > + * For now let's have 4 memory tier below default DRAM tier. > + */ > +#define MEMTIER_ADISTANCE_DRAM (1 << (MEMTIER_CHUNK_BITS + 2)) > +/* leave one tier below this slow pmem */ > +#define MEMTIER_ADISTANCE_PMEM (1 << MEMTIER_CHUNK_BITS) Why is memory type encoded in these values? There is no reason to believe that PMEM is of a lower performance tier than DRAM. Consider high performance energy backed DRAM that makes it "PMEM", consider CXL attached DRAM over a switch topology and constrained links that makes it a lower performance tier than locally attached DRAM. The names should be associated with tiers that indicate their usage. Something like HOT, GENERAL, and COLD. Where, for example, HOT is low capacity high performance compared to the general purpose pool, and COLD is high capacity low performance intended to offload the general purpose tier. It does not need to be exactly that ontology, but please try to not encode policy meaning behind memory types. There has been explicit effort to avoid that to date because types are fraught for declaring relative performance characteristics, and the relative performance changes based on what memory types are assembled in a given system.