From: Naman Jain <namjain@linux.microsoft.com>
To: haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com,
tglx@kernel.org, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org, luto@kernel.org,
peterz@infradead.org
Cc: hpa@zytor.com, riel@surriel.com, mhklinux@outlook.com,
bigeasy@linutronix.de, zhouchuyi@bytedance.com,
houwenlong.hwl@antgroup.com, qq570070308@gmail.com,
bastien.curutchet@bootlin.com, namjain@linux.microsoft.com,
dianders@chromium.org, jgross@suse.com, hawk@kernel.org,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2 0/5] x86/hyperv: Account for lazy mm_cpumask maintenance
Date: Tue, 17 Nov 2026 09:15:02 +0000 [thread overview]
Message-ID: <20261117091507.2088642-1-namjain@linux.microsoft.com> (raw)
Lazy mm_cpumask maintenance leaves CPU bits set after switching to another
address space, with stale memberships removed by native flush callbacks.
Successful Hyper-V TLB flush hypercalls do not run those callbacks, so
historical memberships can persist and cause unnecessary remote flushes.
If the mask covers all present CPUs, the backend selects an all-processor
flush before reaching its per-CPU lazy-TLB checks.
This was found while investigating a regression between Linux 6.8 and 7.0
in SuperBench's eight-GPU PyTorch LSTM training on Azure MI300X VMs.
Recreating DataLoader workers each epoch caused host-to-GPU copy stalls
and timeouts. The reported bisection identified commit 209954cbc7d0
("x86/mm/tlb: Update mm_cpumask lazily"). This is distinct from lazy TLB
mode: a CPU actively running another address space need not be lazy.
Patches 1-3 prepare for target filtering in patch 4 while patch 5 adds
immediate cleanup.
Patch 1 rejects VP-set conversion failures that return zero, allowing
native fallback rather than issuing an empty-target extended hypercall.
Patch 2 makes the native target predicate reusable, retaining its
context-switch ordering and adding a clearly named callback wrapper.
Patch 3 uses native callbacks for requested mask trimming and global ASID
transitions. The latter need guest callbacks to switch CPUs still using
local ASIDs; a Hyper-V invalidation does not do that.
Patch 4 filters targets before selecting either hypercall format or the
all-processors shortcut. Page-table-freeing flushes remain unfiltered.
Patch 5 separately requests best-effort cleanup when no remote target
remains after filtering. Lazy memberships may survive this cleanup.
The cleanup policy remains open for discussion.
The suggested mask-weight > mm_users heuristic was tested as a replacement
for patch 5's approach on a 32-vCPU Hyper-V VM. With three remote readers,
mm_users was 4 while lazy memberships kept mask weights at 26-32. The
heuristic requested trimming for all 10,000 flushes, and each reader
received 10,000 remote callbacks. The scoped approach used Hyper-V
hypercalls instead. A doubled threshold or rate-limited policy has not
been tested.
Earlier testing of the monolithic patch reduced the 32-vCPU all-history
reproducer from 17.235 us to 1.622 us per iteration (two mprotect() calls
and a write). The reporter's application hit the 600-second timeout on the
unpatched 96-vCPU VM and completed in 424 and 436 seconds with the patch.
Changes since v1:
https://lore.kernel.org/all/20261029064234.362385-1-namjain@linux.microsoft.com/
* Split the change into five patches with separate explanations (Dave).
* Add the workload and regression context (Michael).
* Explain the ASID fallback separately from the reported regression (Dave).
* Keep related predicate/wrapper names and comment the definition rather
than the header declaration (Dave).
* Explain target filtering, page-table freeing, and the no-remote-target
cleanup condition. Document the const copy and lazy-membership limit
(Dave).
* Test the mm_users heuristic and retain scoped cleanup as a separate
policy discussion (Dave).
* Omit Fixes tags and stable requests (Dave).
* Rebase onto next-20261116.
Naman Jain (5):
x86/hyperv: Reject unrepresentable TLB flush VP sets
x86/tlb: Make the conditional flush predicate reusable
x86/hyperv: Use native callbacks for trimming and ASID transitions
x86/hyperv: Filter TLB targets before building hypercalls
x86/hyperv: Request mask cleanup when no remote target remains
arch/x86/hyperv/mmu.c | 71 +++++++++++++++++++++++----------
arch/x86/include/asm/tlbflush.h | 1 +
arch/x86/mm/tlb.c | 18 +++++++--
3 files changed, 65 insertions(+), 25 deletions(-)
base-commit: 749a4de370409464cf8501ec5b2451df45c84eb4
--
2.43.0
next reply other threads:[~2026-11-17 9:15 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 9:15 Naman Jain [this message]
2026-11-17 9:15 ` [PATCH v2 1/5] x86/hyperv: Reject unrepresentable TLB flush VP sets Naman Jain
2026-11-17 9:15 ` [PATCH v2 2/5] x86/tlb: Make the conditional flush predicate reusable Naman Jain
2026-11-17 9:15 ` [PATCH v2 3/5] x86/hyperv: Use native callbacks for trimming and ASID transitions Naman Jain
2026-11-17 9:15 ` [PATCH v2 4/5] x86/hyperv: Filter TLB targets before building hypercalls Naman Jain
2026-11-17 9:24 ` sashiko-bot
2026-11-17 9:15 ` [PATCH v2 5/5] x86/hyperv: Request mask cleanup when no remote target remains Naman Jain
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261117091507.2088642-1-namjain@linux.microsoft.com \
--to=namjain@linux.microsoft.com \
--cc=bastien.curutchet@bootlin.com \
--cc=bigeasy@linutronix.de \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=decui@microsoft.com \
--cc=dianders@chromium.org \
--cc=haiyangz@microsoft.com \
--cc=hawk@kernel.org \
--cc=houwenlong.hwl@antgroup.com \
--cc=hpa@zytor.com \
--cc=jgross@suse.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mhklinux@outlook.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=qq570070308@gmail.com \
--cc=riel@surriel.com \
--cc=tglx@kernel.org \
--cc=wei.liu@kernel.org \
--cc=x86@kernel.org \
--cc=zhouchuyi@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.