All of lore.kernel.org
 help / color / mirror / Atom feed
From: Rik van Riel <riel@surriel.com>
To: linux-kernel@vger.kernel.org
Cc: kernel-team@meta.com, dave.hansen@linux.intel.com,
	luto@kernel.org, peterz@infradead.org, bp@alien8.de,
	x86@kernel.org, nadav.amit@gmail.com, seanjc@google.com,
	tglx@linutronix.de
Subject: [RFC PATCH v3 0/7] Intel RAR TLB invalidation
Date: Thu,  5 Jun 2025 12:35:09 -0400	[thread overview]
Message-ID: <20250605163544.3852565-1-riel@surriel.com> (raw)

This patch series adds support for IPI-less TLB invalidation
using Intel RAR technology.

Intel RAR differs from AMD INVLPGB in a few ways:
- RAR goes through (emulated?) APIC writes, not instructions
- RAR flushes go through a memory table with 64 entries
- RAR flushes can be targeted to a cpumask
- The RAR functionality must be set up at boot time before it can be used

The cpumask targeting has resulted in Intel RAR and AMD INVLPGB having
slightly different rules:
- Processes with dynamic ASIDs use IPI based shootdowns
- INVLPGB: processes with a global ASID 
   - always have the TLB up to date, on every CPU
   - never need to flush the TLB at context switch time
- RAR: processes with global ASIDs
   - have the TLB up to date on CPUs in the mm_cpumask
   - can skip a TLB flush at context switch time if the CPU is in the mm_cpumask
   - need to flush the TLB when scheduled on a cpu not in the mm_cpumask,
     in case it used to run there before and the TLB has stale entries

RAR functionality is present on Sapphire Rapids and newer CPUs.

Information about Intel RAR can be found in this whitepaper.

https://www.intel.com/content/dam/develop/external/us/en/documents/341431-remote-action-request-white-paper.pdf

This patch series is based off a 2019 patch series created by
Intel, with patches later in the series modified to fit into
the TLB flush code structure we have after AMD INVLPGB functionality
was integrated.

TODO:
- when flushing multiple CPUs anyway, use RAR to invalidate our own TLB
  (needs a chicken-egg issue with early boot solved)
- some sort of optimization to avoid sending RARs to CPUs in deeper
  idle states when they have init_mm loaded (flush when switching to init_mm?)

v3:
- move cpa_flush() change out of this patch series
- use MSR_IA32_CORE_CAPS definition, merge first two patches together
- move RAR initialization to early_init_intel()
- remove single-CPU "fast path" from smp_call_rar_many
- remove smp call table RAR entries, just do a direct call
- cleanups suggested (Ingo, Nadav, Dave, Thomas, Borislav, Sean)
- fix !CONFIG_SMP compile in Kconfig
- match RAR definitions to the names & numbers in the documentation
- the code seems to work now
v2:
- Cleanups suggested by Ingo and Nadav (thank you)
- Basic RAR code seems to actually work now.
- Kernel TLB flushes with RAR seem to work correctly.
- User TLB flushes with RAR are still broken, with two symptoms:
  - The !is_lazy WARN_ON in leave_mm() is tripped
  - Random segfaults.


             reply	other threads:[~2025-06-05 16:35 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-05 16:35 Rik van Riel [this message]
2025-06-05 16:35 ` [RFC PATCH v3 1/7] x86/mm: Introduce Remote Action Request MSRs Rik van Riel
2025-06-05 16:35 ` [RFC PATCH v3 2/7] x86/mm: enable BROADCAST_TLB_FLUSH on Intel, too Rik van Riel
2025-06-05 16:35 ` [RFC PATCH v3 3/7] x86/mm: Introduce X86_FEATURE_RAR Rik van Riel
2025-06-05 16:35 ` [RFC PATCH v3 4/7] x86/apic: Introduce Remote Action Request Operations Rik van Riel
2025-06-05 16:35 ` [RFC PATCH v3 5/7] x86/mm: Introduce Remote Action Request Rik van Riel
2025-06-05 18:54   ` Nadav Amit
2025-06-05 19:40     ` Rik van Riel
2025-06-05 22:45       ` Nadav Amit
2025-06-06  0:03         ` Rik van Riel
2025-06-06 19:58           ` Nadav Amit
2025-06-05 16:35 ` [RFC PATCH v3 6/7] x86/mm: use RAR for kernel TLB flushes Rik van Riel
2025-06-05 16:35 ` [RFC PATCH v3 7/7] x86/mm: userspace & pageout flushing using Intel RAR Rik van Riel
2025-06-05 16:50 ` [RFC PATCH v3 0/7] Intel RAR TLB invalidation Sean Christopherson
2025-06-05 16:58   ` Dave Hansen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250605163544.3852565-1-riel@surriel.com \
    --to=riel@surriel.com \
    --cc=bp@alien8.de \
    --cc=dave.hansen@linux.intel.com \
    --cc=kernel-team@meta.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=nadav.amit@gmail.com \
    --cc=peterz@infradead.org \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.