From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E623BC77B7C for ; Mon, 23 Jun 2025 16:50:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87DEF8D0008; Mon, 23 Jun 2025 12:50:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 855E18D0005; Mon, 23 Jun 2025 12:50:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 792398D0008; Mon, 23 Jun 2025 12:50:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6C8968D0005 for ; Mon, 23 Jun 2025 12:50:09 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3CB60160A73 for ; Mon, 23 Jun 2025 16:50:09 +0000 (UTC) X-FDA: 83587252938.03.3F9212A Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf17.hostedemail.com (Postfix) with ESMTP id A796D4000A for ; Mon, 23 Jun 2025 16:50:07 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CSkFswit; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750697407; a=rsa-sha256; cv=none; b=34EwioBmADuAC33S+5PjZ2NnB2ezfFs7WCrLdzM6LPWX8Xgx5Jfawcn85gyI+yCnwu5+yk qlm7N0CQgw7Rd8e1DxjS6P79FfgaPPbh72qY0yM13M9EJ667GVgWJY1ffIFDqQ4e28y8ZI dmt3VNzOpH/cI07z6m+JbeGkecLlfpE= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CSkFswit; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750697407; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sb2yHxSxL+pQgPO6enq6v0ZUPYI306kjUESw+dvzxxs=; b=3IJZLGBjOWQCZHkCGzih35JugHf3DpIpm2t4CTGjStN6dhzoECzdH8ZUGkqKIxnrgoEadz S56UE5oVFoHeOub4DsnxycaqGsiHTGFSxoLhSzdqRgtltKJqKQi8lUBaD4uUiezTMxFpbM 2b0dX1IjUaWRtJbIXtm5ERb4Qc0Soyw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id AC16F6112C; Mon, 23 Jun 2025 16:50:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36BE3C4CEEA; Mon, 23 Jun 2025 16:50:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750697406; bh=ixzNiYvSohABEP65EJQJpxD1tvXomrgSMaFRGU3G4rE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CSkFswit9LNxWkHNZYRaVCPYGDIC4iLRpVVVwTIJo/1jVx/Kh4FPassaWhLJBLYbF fMk8Wn0unQumPpjSqdYdOBBiJYVoAzvZdfWRQqv9EjSAx5Upbid6JWDW56KZu73foU ha24bwGZLbCq/KhjnMM7ag4/nfxS7L3H+6HVaql+v4EmAaGfPg0Hv+AXqMKCiDJIUT 2v9c6ShuwW+DG63DNnh/ZRSCUtLTexj7M/9bT0Un+fx0T/EFxiZ3cdnZbCv3IsIDZd X/8726K3K7T4d5IZpkXfG76TZ5RaKy+6zHWp3M0rnPCbVPFJ9ynTaoVYEB+rpNLZhR LEaO5vymAty/A== From: SeongJae Park To: Joshua Hahn Cc: SeongJae Park , Bijan Tabatabai , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, david@redhat.com, ziy@nvidia.com, matthew.brost@intel.com, rakie.kim@sk.com, byungchul@sk.com, gourry@gourry.net, ying.huang@linux.alibaba.com, apopple@nvidia.com, bijantabatab@micron.com, venkataravis@micron.com, emirakhur@micron.com, ajayjoshi@micron.com, vtavarespetr@micron.com Subject: Re: [RFC PATCH v2 2/2] mm/damon/paddr: Allow multiple migrate targets Date: Mon, 23 Jun 2025 09:50:04 -0700 Message-Id: <20250623165004.43394-1-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250623140808.2479244-1-joshua.hahnjy@gmail.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: A796D4000A X-Stat-Signature: b7wcujbwe4wpxj7ta6qzme8rnqyoegeo X-Rspam-User: X-HE-Tag: 1750697407-232655 X-HE-Meta: U2FsdGVkX19kFv2yxwEGXRxzO9SaVQK8KPsHhbsGFitIILmAU5GOJrQpwOZ2GkRrgKEGvYNbkitsZrOxUY+1egVX8KoNl7dBCVELjXzXWUIgdHOHBHNfq9xcOHkXHXnjbnxiJraBS/tSPsmN9U0sMMEcEatecaQeQhIcyRs87D9WkRmCxd9Ch1Nd1qlqbWHD/VGSUIF4Bq3hDHtUvkLSiOI4ct/1Va2FlQykkHvAjLvtqfW5wINBPLdMiNGb5INWu+OcySzMFY7uzMqSZ1r2gNy0i7BSMSpUHun4fvgOdpB2iQEPKiLJXQVViF1aMp4bsiHQ9QLtuD2CSY9uuvqAt6yuQ25gKoS3rq7Li935WbFlcna+wAYKTIByJEColK4nkyg659449R7tUzsa+G+n+vHVPCZs/FKE5pJcaxNeruImUTPH6arEbReNCRHG77NuHWeBxVF8k4MkvbkiV78pZg/Gny6bEoVTdxViJLp/sxzF03onrJWXLF1Bj6iSzXHOvOteFrigtZLYrCmxOcDv6FT6nQjYJgjT3r5x9PS7Wi8UEPa//kefX/tOvRzoHUkge1KMzW5X3WGYpo+HDyUzVq/CDuIw/IZxDlPGCQbMWiMBVOmocnGGbQ8KH/wm9dbMtlJSaGMNi0NZD9qDO5SYYgTBQNr7z1g53IYSmKaqPM0w3s6onGQ7X0RwwuJZhext7s4FHFyTCvU8wW6gstx6VYTS0O9BS6ba0SD5DGjUm8lqmJOOk1VZa+KDsZEMnmfiPXT8EXNe3uKso3PrVNpxf6RvD4nISUpqfuQVzYKlKfX8nyhkiLMAICdGbO1+E/4QgWSOMjvlkzJJNvKYmrpCFGAMiM/apjEHAF0e2tG31vmnSYAvm4UZv84Jtn44ovRd2pIk8VNaIfe8bObbfshvPJJE+AWutGcbRxwM8WvBhBv9rIj9N9et0S/7Np65Isaf8xPnNQTVS6ONk5l5m3y KU6DJTv3 KpFEAc3ZheHk8Vcp/q+LDByOQjyFi+sDI6uAWeFAGWOGj32uIuM2j2HbBBjj/Uockzp51LApBXtr8CiXctlQsFNSoRMB3qt/bE1VjlfTeS7qEMSM2V3JjKEr8jILcZJih6B6y4sn0Hib1RqWll6/zj6qgJ/bT6VSJOUvpM0Zh1Z0lFxFh3jPKdHqVS11mbPCkU+7XxAVIKWHQAOL2X4ULgLaeFd9TKLImVAynvXr3VnHAniejRFGjC20bZjl1L0OrbVxo7KQi/bBJ8giGwHz+GZkD2rhAOtxJ9GCBAw366OGn1Q3rFlI5OCCN67mgFEyWNLkhb168fSSORr0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 23 Jun 2025 07:08:07 -0700 Joshua Hahn wrote: > On Sat, 21 Jun 2025 11:11:27 -0700 SeongJae Park wrote: > > > On Sat, 21 Jun 2025 11:02:15 -0700 SeongJae Park wrote: > > > > [...] > > > I'd hence suggest to implement and use a simple weights handling mechanism > > > here. It could be roud-robin way, like weighted interleaving, or probabilistic > > > way, using damon_rand(). > > > > > > The round-robin way may be simpler in my opinion. For example, > > [...snip...] > > > Actually, probabilistic way may be not that complicated. Maybe we could to > > below here. > > [...snip...] > > > But damon_rand() might be more expensive than the roud-robin way, and arguably > > roud-robin way is what usrs who familiar with weighted interleaving may easily > > expect and even prefer? I have no preferrence here. > > Hi SJ, > > If you have no preference here, I would like to add some thoughts : -) > [...] > I think that code complexity aside, round-robin may be the better choice for > a few reasons. Like you mentioned, I think it is what users might be used to, > if they are coming from weighted interleave code. Also, I think a round-robin > way will prevent worst-case scenarios where we get a long stretch of allocations > on the "wrong" node (but maybe this isn't a big deal, since it is so unlikely). > > Finaly -- If we run workloads with mempolicy wet to weighted interleave > *and* with the weights already set, then pages will be allocated in a > round-robin fashion. I think it may be best to try and minimize migration costs > by trying to keep these weights in-sync. That is, if we have a 2:1 ratio, > we will have the following allocation: > > node0 | oo oo oo oo oo oo oo ... > node1 | o o o o o o ... > > Using a probabilistic migration, it might change the pattern: > > node0 | oooo oo o ooo oo ... > node1 | oo o o o o ... > > That is, the ratio might be preserved, but we may be doing unnecessary > migrations, since a probabilistic allocation isn't aware of any underlying > patterns. With a round-robin allocation, we have a 1/total_weight chance that > there will be no additional migrations, depending on where the round-robin > begins. I also want to note that weighted interleave auto-tuning is written > to minimize total_weight. > > I'm wondering what you think about this. Perhaps there is a way to know where > the "beginning" of round-robin should begin, so that we try to keep the > allocation & migration pattern as in-sync as possible? I have a suspicion > that I am way over-thinking this, and none of this really has a tangible > impact on performance as well ;) The theory makes sense to me. I also not very sure how much visible difference it will make on large scale real workloads, though. Since at least the theory makes sense and we show no risk, I think taking the round-robin appraoch would be a saner action, unless we find other opinions or test results. > > Thank you as always SJ, have a great day!! Thank you, you too! Thanks, SJ [...]