Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Qiliang Yuan <odys.yuan@gmail.com>
To: "Huang, Ying" <ying.huang@linux.alibaba.com>,
	David Hildenbrand <david@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Zi Yan <ziy@nvidia.com>, Matthew Brost <matthew.brost@intel.com>,
	Joshua Hahn <joshua.hahnjy@gmail.com>,
	Byungchul Park <byungchul@sk.com>,
	Gregory Price <gourry@gourry.net>,
	Alistair Popple <apopple@nvidia.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 1/2] mm/migrate: walk runs of consecutive pages in do_pages_stat_array()
Date: Thu,  8 Oct 2026 10:27:48 +0800	[thread overview]
Message-ID: <20261008022748.3349345-1-odys.yuan@gmail.com> (raw)
In-Reply-To: <87v77ggu14.fsf@DESKTOP-5N7EMDA>

Hi Ying, David,

On Mon, 05 Oct 2026 21:24:07 +0800, Huang, Ying wrote:
> As pointed out by David, nanosecond-level optimization for a not-so-hot
> path isn't very attractive.  I understand the target of your
> optimization is not the performance of a single page but that of a large
> number of pages (such as 16 GiB).  So, please describe more clearly why
> your change is necessary, for example, by providing the performance
> improvement of querying 16 GiB memory.

Right, the target is large buffers, which also answers David's question
about the need. Mooncake, the KV-cache store used for Kimi serving,
calls move_pages() with a NULL node list on every 4K page of the
buffers it registers for RDMA, to find which NUMA node each one lives
on. Its issue tracker reports 216 ms for a 4 GiB buffer, and it
registers hundreds of GiB per instance, so this query alone takes
seconds of every startup.

Time to query every page of a populated buffer, with the two v2
patches applied, median of 5 runs:

                       before      after      change
  1 GiB,  4K pages     27.3 ms     3.2 ms     -88%
  4 GiB,  4K pages     109.5 ms    13.0 ms    -88%
  16 GiB, 4K pages     486.1 ms    52.3 ms    -89%
  1 GiB,  THP          23.5 ms     0.5 ms     -98%
  4 GiB,  THP          95.0 ms     2.1 ms     -98%
  16 GiB, THP          381.0 ms    9.1 ms     -98%

> Additionally, the raw performance number depends on the system under
> test.  Please provide a little more information about your testing
> system, for example, the CPU architecture, generation, physical core
> count, etc.  For comparison, the performance improvement percentage
> would also be helpful.

The host is a 2-socket AMD EPYC 9654 (Zen 4, 96 cores per socket). The
test runs in a KVM guest on 7.3-rc5 with 16 vCPUs pinned to one host
NUMA node and 32 GiB of memory, split into two guest NUMA nodes. The
test program is bound to CPU 0, and the before and after kernels were
measured back to back in the same guest.

Thanks,
Qiliang


  reply	other threads:[~2026-10-08  2:27 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-02  1:25 [PATCH v2 0/2] mm/migrate: speed up move_pages() node queries Qiliang Yuan
2026-10-02  1:25 ` [PATCH v2 1/2] mm/migrate: walk runs of consecutive pages in do_pages_stat_array() Qiliang Yuan
2026-10-02 18:53   ` David Hildenbrand (Arm)
2026-10-05 13:24   ` Huang, Ying
2026-10-08  2:27     ` Qiliang Yuan [this message]
2026-10-02  1:25 ` [PATCH v2 2/2] mm/migrate: raise the do_pages_stat() chunk to 512 pages Qiliang Yuan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261008022748.3349345-1-odys.yuan@gmail.com \
    --to=odys.yuan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=apopple@nvidia.com \
    --cc=byungchul@sk.com \
    --cc=david@kernel.org \
    --cc=gourry@gourry.net \
    --cc=joshua.hahnjy@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=matthew.brost@intel.com \
    --cc=ying.huang@linux.alibaba.com \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox