From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9725DC61CE8 for ; Thu, 12 Jun 2025 18:15:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02A166B009A; Thu, 12 Jun 2025 14:15:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F1D4E6B009B; Thu, 12 Jun 2025 14:15:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0C276B009C; Thu, 12 Jun 2025 14:15:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BEA5B6B009A for ; Thu, 12 Jun 2025 14:15:00 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5C070141D05 for ; Thu, 12 Jun 2025 18:15:00 +0000 (UTC) X-FDA: 83547549960.05.03CFE5D Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) by imf05.hostedemail.com (Postfix) with ESMTP id 8E1EC100002 for ; Thu, 12 Jun 2025 18:14:58 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fJQu4xgg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of bijan311@gmail.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=bijan311@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749752098; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=7dGCBQnCx0nj6LJC8ipSZAzvXRddXH3Z76gYMfvl3qo=; b=rDYvMr9VyVt9HuSvgCNlSjbNSnqVU0CStZZP+msf9KPriJkzSvULdeNYPpKU0FsqRJgsQS FwdIs+FmgAo3ffpOfnAz6U5zYs/OH4BIrWa8x/M+ZFOEJTmpSXLrL2FMD11zmYeUmEl5FD Aq+AqA+ipxvafpJc0nbmL41Jrcf1TM0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749752098; a=rsa-sha256; cv=none; b=Dq9xymFatD+D8g5TfoEg1b59y0H8B46bdxx9sXALFHRX4KpmZ0Wx+GOhjXWcPZqFcmA/ch jsHj8qUZ9KkdNzTrh6Kv5FOy5T5M6mk2rdH4IB05TQLGC84sBRGMtG8JvdMuadWgUgM+1J E5Ay6x87L82xa0FpnJoSHyOIkStlExQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fJQu4xgg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of bijan311@gmail.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=bijan311@gmail.com Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-e81f8679957so1176837276.2 for ; Thu, 12 Jun 2025 11:14:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749752097; x=1750356897; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=7dGCBQnCx0nj6LJC8ipSZAzvXRddXH3Z76gYMfvl3qo=; b=fJQu4xggvnR8fhj1GNqT5ggOazVjTUU23ggjOA9zbwfMr0yGIJcqF2vQDb3IXL7ORi XJwcCFEmd7wlFcBC9fVGVk5WlPErtyNlnFrETp8LuRne9FriIyqwppIZeLmD5ptF2flC N6/UEgNH4dkBAwUgWxLzAYbwa5LW1Y6YQGT+dQ3YoKAC5pxHv7399PuoErWK4qzRlofN CXOYB6EEdgeSxMp4X40p+49hEnGXqnwo4G3V5uZgrmyheTZiRnV6I7+DE0qAVqG9iOOS GbpI3vV3SGBzlxhobGsvMrfQ1QaTh7XI1II3yv2HlAgQuFbc3NvzgL0boqjhDR9UiwBk pNrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749752097; x=1750356897; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7dGCBQnCx0nj6LJC8ipSZAzvXRddXH3Z76gYMfvl3qo=; b=YH+rPkNea2VL/cUCjBLi32fGRAR7BjE88J/34jaGrWhUHIvX///ooc4EuQ7y97eYA3 GQYsZtCa4WHLhLHr/OX+34Zpx7xuuPDy8aRZ0IaB7jNun+I6mYGODyZBY75gCjG7JxO/ BbMGTc/ESDXmVGwzqHoHolsXNl6rDp8TF3AaI+gCnxCozBgky9v9S4NJVQ6UHQHNMg7P pXoR/dG5p7tix1E9+DzyvGJaiqLzb0AEJfizzM6ipY8fM2CjQBEyVUoIPDkN0sKUZ4L3 p7pTD2bXpQdgzytbj/6/+dngEfzKiMG7w2jOAyLtd2AsJ44RQ2/AOzldCjK1kEQ66FrU O7WA== X-Forwarded-Encrypted: i=1; AJvYcCXFeWfsXOpFQ+qjO+a7EAQ9S6BB28UJXKeXO7TWqdHMaOz83uj3s3+QvCRoVX9wJVLux3JZk+SAiw==@kvack.org X-Gm-Message-State: AOJu0YxLxULnSnNzAtWhqrzGuXY42kJ+obzjO26HjIzfciu63AevHcNc JfBFdYKWBOoSXSeZkHpCzydNBc6ipg2j9QacHEQh13DBU3qLYUfAArKP X-Gm-Gg: ASbGncsi6s+rRhlmNBld0G6Qb+9CelYxmHt5KyE3pxVAsAAA27ws9GTZUAvCOhD+Nil sBm45iPs9K3YC9gWEC0azUfLjEJ1Zfqxyk9v77RncpPk5GL/yFNkDIIdykWlI3s7m+2eZArqOJW oamqaXE9HpxjMI/iMcqhGt+P72qYzdXmjfwtJOGVtatCwmJqn/OjuG8uh4CeXITrZi2iqBsr18t qiQ2Hd/5uo7rHc9dw7cgul7n4SQPm7k0nkEj8SQTvxwCVfjR1lakVvd2venvt4+x6nXqnrK557M p3kOVCekL0XtMzGN4Rj31UJhpMFzMLIGXHcZlptvNh+/Lca8U7Z+3f/hLBjgcQpukdlOLnZZuIf CCY6LNkCRiHj+Ctydjg== X-Google-Smtp-Source: AGHT+IHuilM8OKD/FKaBak0BosxjVKmpehSXwuiIqtTuQ6Jsg2kCbvgxdn/IU8iSp6RJyUJ/a8SdjA== X-Received: by 2002:a05:6902:15c1:b0:e82:61:6536 with SMTP id 3f1490d57ef6-e821a74a74bmr951361276.7.1749752097431; Thu, 12 Jun 2025 11:14:57 -0700 (PDT) Received: from bijan-laptop.attlocal.net ([2600:1700:680e:c000:dd1b:d4ae:15de:11db]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e820e312452sm592480276.40.2025.06.12.11.14.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jun 2025 11:14:57 -0700 (PDT) From: Bijan Tabatabai To: damon@lists.linux.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: sj@kernel.org, akpm@linux-foundation.org, corbet@lwn.net, david@redhat.com, ziy@nvidia.com, matthew.brost@intel.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, byungchul@sk.com, gourry@gourry.net, ying.huang@linux.alibaba.com, apopple@nvidia.com, bijantabatab@micron.com, venkataravis@micron.com, emirakhur@micron.com, ajayjoshi@micron.com, vtavarespetr@micron.com Subject: [RFC PATCH 0/4] mm/damon: Add DAMOS action to interleave data across nodes Date: Thu, 12 Jun 2025 13:13:26 -0500 Message-ID: <20250612181330.31236-1-bijan311@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: jupzaia9x6emekhg16kh7e8dga4f1urj X-Rspamd-Queue-Id: 8E1EC100002 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1749752098-28734 X-HE-Meta: U2FsdGVkX180s65LAubM5iFJ7AT4nZKKK8sdKVAKVlC8PWmJ+5g1gjlQfXG2OqBRNdoVo2JAS4SmbqUFxOVhkVN1aE9afz/uyI5IiVOTO3yMbRo+t99DZUiNABB0/7awkSrpSJUb4MqmHpDU+L/GDSehGo48UZhK5VoxnXVNWhs4a5N2a0REU974E5aH/8X1UGLmARnTGMvni+nsafN1ToGU3a+7qlRfvPykpsmS3CgPMZG6tqjfBGbnsfsxDZqEfcJLoiNpF+XbZrj/73nZcbS5gCsTomM3GsYfzAVXlqxQY8frDGedf0t+4SvFKpW2UeZ5oJDE1BZ0TcyODf026XhohHlRSM6mR0GVUw4SL3XBTXrmI4rG8XvL74Xiwjn2GWDLKNgfdIGTKaQDeYFyBu5NNVexjhhd+KBCXY1DPlrv36nX3m1zEm1mBpSYwfkNfv/N81qGd4RF4s2M9btdMoHPE6XuOU9jYVuvr82ALTez6IqsCgMD8SFilBdTF9u9HVkcZ+SthxPAJ7fMcM82dzxeWTPJmm7RzxS8QL03vzMBDvz1WgsExQUSvJBpQ8mRts5nHFZixmU2Cu6Bgj124lnf+GBMlAgATSmwX9PrhbKMeG61qm3lLjQ4pCKOSMxD4/nNq01dRKlFyMYjlrc0TXgpfdWlEiFF3f1dE7tlqNQMMHwKAcvKk16da2fk5ZUOromTOT1HHGU4DKvFfbbec3lO+W7FRLFH2jdwYWtJoc/fxIabAcu8nzYvTPIUz1eKLG9iZr7VsDPLC4w32ScVTlXIM43exV/zMTh9oKyD6rRPFsomROUoonOyLVUputp1O0ZPfK/u+t1fDZt8kvB5KK0Pz6EdqNCMPE0gEqQNBFqfDXR3Z3xm0Iqx+HalIlhVpo7zzvJkVNgerAIRCmDbjxLjsRYfmj/4RFDPwsVvpapOrjcB4jufEHGBHdssfJ99u0sCFj20j+F1GScQJYa aWQLGO/c q/fixVZKWDfX+vWuWx0HBU6qZdUhhE/x4PdCqoNmeX7TxcMKyvaTCilA9oLIfvtUXkgfZsUQKGEEXMPyQ8twud6qlaZEskbx3g4lq36TlOJvpBu1eA/5PTtqLwK38KZwLpeFNAV0wAWOJnVkM8/AuyjC1Ecy/SWDguMFQ0CrCi07LtsiNfCi1Yc5+YivPB30g+cj/d1ZzdqrP/rCs8KvBuAvDllECaMEbmZiPlDp7l7YLG9yWFzsr2eMZ/a/obRgFVDllratOPxTe/lDsS3OQpwt8eShNW0xn1lI9NCFModg/CqaE0Smz0zFRkxU2dlCXnWVaNOXObB3S0D8S8ZtD4YrrC7bEGa7+vb17zN34pP0rPypr4/1tF/JxCaJEt0e9NBjm4TwqhhcVHyLNwYIYk++BvLyIdjywbhrb3Nv0XIr+1CVOJ2VimGCsEhHhSuUhoFVOHK0U5oV4cC9b++Aj+jawzPsHSfnmsUBhhBwwWAMRycKTSQOVpvLkGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Bijan Tabatabai A recent patch set automatically set the interleave weight for each node according to the node's maximum bandwidth [1]. In another thread, the patch set's author, Joshua Hahn, wondered if/how these weights should be changed if the bandwidth utilization of the system changes [2]. This patch set adds the mechanism for dynamically changing how application data is interleaved across nodes while leaving the policy of what the interleave weights should be to userspace. It does this by adding a new DAMOS action: DAMOS_INTERLEAVE. We implement DAMOS_INTERLEAVE with both paddr and vaddr operations sets. Using the paddr version is useful for managing page placement globally. Using the vaddr version limits tracking to one process per kdamond instance, but the va based tracking better captures spacial locality. DAMOS_INTERLEAVE interleaves pages within a region across nodes using the interleave weights at /sys/kernel/mm/mempolicy/weighted_interleave/node and the page placement algorithm in weighted_interleave_nid via policy_nodemask. We chose to reuse the mempolicy weighted interleave infrastructure to avoid reimplementing code. However, this has the awkward side effect that only pages that are mapped to processes using MPOL_WEIGHTED_INTERLEAVE will be migrated according to new interleave weights. This might be fine because workloads that want their data to be dynamically interleaved will want their newly allocated data to be interleaved at the same ratio. If exposing policy_nodemask is undesirable, we have two alternative methods for having DAMON access the interleave weights it should use. We would appreciate feedback on which method is preferred. 1. Use mpol_misplaced instead pros: mpol_misplaced is already exposed publically cons: Would require refactoring mpol_misplaced to take a struct vm_area instead of a struct vm_fault, and require refactoring mpol_misplaced and get_vma_policy to take in a struct task_struct rather than just using current. Also requires processes to use MPOL_WEIGHTED_INTERLEAVE. 2. Add a new field to struct damos, similar to target_nid for the MIGRATE_HOT/COLD schemes. pros: Keeps changes contained inside DAMON. Would not require processes to use MPOL_WEIGHTED_INTERLEAVE. cons: Duplicates page placement code. Requires discussion on the sysfs interface to use for users to pass in the interleave weights. This patchset was tested on an AMD machine with a NUMA node with CPUs attached to DDR memory and a cpu-less NUMA node attached to CXL memory. However, this patch set should generalize to other architectures and number of NUMA nodes. Patches Sequence ________________ The first patch exposes policy_nodemask() in include/linux/mempolicy.h to let DAMON determine where a page should be placed for interleaving. The second patch implements DAMOS_INTERLEAVE as a paddr action. The third patch moves the DAMON page migration code to ops-common, allowing vaddr actions to use it. Finally, the fourth patch implements a vaddr version of DAMOS_INTERLEAVE. [1] https://lore.kernel.org/linux-mm/20250520141236.2987309-1-joshua.hahnjy@gmail.com/ [2] https://lore.kernel.org/linux-mm/20250313155705.1943522-1-joshua.hahnjy@gmail.com/ Bijan Tabatabai (4): mm/mempolicy: Expose policy_nodemask() in include/linux/mempolicy.h mm/damon/paddr: Add DAMOS_INTERLEAVE action mm/damon: Move damon_pa_migrate_pages to ops-common mm/damon/vaddr: Add vaddr version of DAMOS_INTERLEAVE Documentation/mm/damon/design.rst | 2 + include/linux/damon.h | 2 + include/linux/mempolicy.h | 2 + mm/damon/ops-common.c | 136 ++++++++++++++++++++ mm/damon/ops-common.h | 4 + mm/damon/paddr.c | 198 +++++++++++++----------------- mm/damon/sysfs-schemes.c | 1 + mm/damon/vaddr.c | 124 +++++++++++++++++++ mm/mempolicy.c | 4 +- 9 files changed, 360 insertions(+), 113 deletions(-) -- 2.43.5