Re: [PATCH] mm/damon/vaddr: avoid unnecessary migration between target nodes

[SeongJae Park <sj@kernel.org>]

On Mon,  4 Aug 2025 10:19:20 -0500 Bijan Tabatabai <bijan311@gmail.com> wrote:

> On Mon, 4 Aug 2025 21:46:49 +0900 Akinobu Mita <akinobu.mita@gmail.com> wrote:
> 
> > Hi,
> > 
> > 2025年8月4日(月) 3:01 SeongJae Park <sj@kernel.org>:
> > >
> > > Hello Akinobu,
> > >
> > > On Sun,  3 Aug 2025 22:00:28 +0900 Akinobu Mita <akinobu.mita@gmail.com> wrote:
> > >
> > > > This change avoids unnecessary migration by checking if the folio to which
> > > > the migration is applied is already located in one of the target nodes.
> > >
> > > Each folio will get a single target node among the target nodes, based on the
> > > weights of targets and the virtual address of the folio.  So skipping the
> > > migration because the folio is on one of target nodes can result in wrong
> > > behavior (not doing required migrations)?
> > >
> > > Also, a similar purpose optimization was already made by Bijan.  I guess you
> > > aware of that and therefore Cc-ed Bijan.  Could you further share differences
> > > between this and Bijan's one, and measured additional performance gain from
> > > this patch, if you have?
> > >
> > > [1] https://lkml.kernel.org/r/20250725163300.4602-1-bijan311@gmail.com
> > 
> > In my current DAMON setup, the target nodes for migrate_hot action
> > are 0-3, and for migrate_cold action, they are 4 and 5.
> > 
> > I'm looking for a way to prevent unnecessary movement between target
> > nodes for each action, making interleaving an optional choice. For
> > instance, I'd like to avoid migrations from node 4 to node 5 during a
> > migrate_cold action.
> 
> Where a folio is placed is based on the virtual address of the folio and
> the folio's offset from the start of the VMA it is in. So, as long as the
> interleave weights and the folio's offset in the VMA are the same, folios
> should not be migrated after being placed. This placement algorithm is the
> same as the one used for mempolicy weighted interleaving.
> 
> It sounds like you are keeping the interleave weights constant, and I would
> imagine that the virtual addresses the folios are mapped to are constant,
> so I'm guessing the start address of some VMAs are changing. I don't think
> I've seen the bahvior your describing in the workloads I've run, but I
> guess it's possible if VMAs are being merged or broken up a lot, such as
> through frequent mmaps/munmaps. Do you think something like this is
> happening in your workload?
> 
> How often are the undesired migrations happening? Every time the scheme is
> applied?
> Also, would you be willing to share how you are configuring DAMON?

I also want to learn more about your setup, use cases, and measured amount of
problem and benefit if you have.

> 
> > I've observed a difference in behavior compared to Bijan's patch, and
> > it seems there's a need for both approaches. Would it be possible to
> > introduce a new option that allows us to specify whether interleaving
> > is forced or optional, and then adjust the behavior based on that
> > setting?

We can add options if it is promising.  I have no good idea about if this is
promising or not for now.  It would be helpful if you could share more details
about your use cases and measurements.

> 
> I wonder if this would be better handled as a paddr scheme. That way
> you can configure the schemes such that only physical addresses in nodes
> 4-5 will migrate pages to nodes 0-3, and only physical address in nodes 0-3
> will migrate to nodes 4-5.

Agreed.  If Akinobu's use case is not depend on virtual address space, I think
paddr is a feasible direction to go.  Note that 'paddr' doesn't support
multiple destination nodes for now, so development of that would be needed if
we decide to go in this way.  But, again, I'd love to learn more about
Akinobu's use case and the problem first.


Thanks,
SJ

[...]