Re: Question about the "TLBs and I-cache are private to each vCPU" guarantee with VTTBR_EL2.CnP

[Marc Zyngier <maz@kernel.org>]

On Mon, 06 Jul 2026 09:25:46 +0100,
Tangnianyao <tangnianyao@huawei.com> wrote:
> 
> 
> 
> On 7/6/2026 15:25, Marc Zyngier wrote:
> > On Mon, 06 Jul 2026 04:30:30 +0100,
> > Tangnianyao <tangnianyao@huawei.com> wrote:
> >>
> >>
> >> On 7/6/2026 1:28, Wei-Lin Chang wrote:
> >>> Hi,
> >>>
> >>> Let me try to answer this:
> >>>
> >>> On Sat, Jul 04, 2026 at 03:45:56PM +0800, Tangnianyao wrote:
> >>>> Hi, all
> >>>>
> >>>> I'm trying to understand the TLB and I-cache invalidation in
> >>>> `kvm_arch_vcpu_load()` that is intended to "guarantee that both TLBs and
> >>>> I-cache are private to each vCPU".
> >>>>
> >>>> As I understand it, when `VTTBR_EL2.CnP == 1`, `__kvm_flush_cpu_context()`
> >>>> only performs a local TLB and I-cache invalidation, which does not seem
> >>>> sufficient to guarantee that property.
> >>>>
> >>>> In fact, even if the invalidation were extended to the Inner Shareable
> >>>> domain, it still seems difficult to guarantee “TLBs and I-cache are
> >>>> private to each vCPU”, when `VTTBR_EL2.CnP == 1`, as long as multiple
> >>>> vCPUs from the same VM may be running concurrently on different PEs.
> >>> I think you have missed that when 2 stages are involved, both stages
> >>> have to set CnP == 1 in order to share TLB entries (Arm ARM R_ZVRZW).
> >>> So if TLB entry sharing happens, the guest kernel must have allowed it
> >>> in the first place (by setting TTBR0/1_EL1.CnP == 1), hence accidental
> >>> sharing that you are worried about won't happen.
> >>>
> >>> __kvm_flush_cpu_context() is solving problems that occur when multiple
> >>> vCPUs of a VM are multiplexed on a single physical CPU.
> >> Thanks for you answer.
> >>
> >> If guest kernel allow TLB shared across CPUs by setting TTBR0/1_EL1.CnP == 1,
> >> does kvm still need to guarantee that TLBs are private to each vCPU?
> > Yes, because there is nothing that describes which physical CPUs
> > actually share TLBs. So the only possible course of action is to
> > ignore what the guest says and fallback to something that is safe.
> >
> >>>> So I have two questions:
> >>>>
> >>>> 1. What is the rationale behind the comment that "guarantee that both TLBs
> >>>> and I-cache are private to each vCPU"?
> >>> I assume you are asking why keeping both TLBs and I-cache private per
> >>> each vCPU is required. The fundamental answer is that each physical CPU
> >>> is expected to have its own TLB and I-cache, so we must uphold that
> >>> property for vCPUs as well. vCPUs can be scheduled on the same physical
> >>> CPU, and use the same physical TLB/I-cache, obviously, so extra
> >>> invalidations need to be done.
> >> Let's assume that both Stage-1 CnP and Stage-2 CnP are enabled.
> >>
> >> As I understand it, the architecture permits TLB to be shared by multiple
> >> PEs within an Inner Shareable domain. Right?
> >>
> >> If an implementation allows TLB entries to be shared in this way, it seems
> >> that the current invalidation performed by kvm would no longer be sufficient
> >> to guarantee that TLBs are private to each vCPU.
> > Care to explain why?
> >
> > The core assumption is that a TLBI take effect on all the PEs the TLB
> > is shared with. If this doesn't work, then CnP is unusable, because it
> > is then impossible to guarantee that a translation will be refetched
> > (you could always hit in another PEs TLBs). Such an implementation
> > would be terminally broken.
> >
> > 	M.
> >
> For example:
> Sharing the TLB between the two SMT threads of the same physical core can
> reduce hardware cost while increasing the effective TLB coverage.

I have a precise idea of what TLB sharing can achieve.

> A local TLBI take effect on the whole TLB shared by the two SMT threads,
> with the sharing enabled by CnP.
> 
> In this scenario, enabling CnP in KVM appears to break the guarantee that
> TLBs are private to each vCPU when multiple vCPUs of the same VM run
> concurrently on different SMT threads of the same PE.

Please stop inventing your own terminology. A PE *is* a thread in the
architecture. There is no such thing as "threads of the same PE".

But more to the point: if TLBI invalidates the relevant TLBs for all
the PEs that share them, *why* isn't the current KVM behaviour not
enough to ensure that the vcpu will not hit old TLBs that are there as
a result of a vcpu having run there previously?

Please explain.

	M.

-- 
Without deviation from the norm, progress is not possible.