From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99D2CC43458 for ; Mon, 6 Jul 2026 14:31:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:CC:To: Subject:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LEJv5GyWiRmG2lSUuloOO2+UEipCg7jbRxfrhXUgk8s=; b=QeE4sCSCSNKvV4he69oiSX4HcU T9Tng2N1E0xSkOGdjbPVc3dxiQYsp+QxfNUDTnZtPlVm4bYFTZA4pj90oHdNDpE+t13G36BkLngCa vfyRVr96vus5FEdji4Xq6LnMTyF+90MJedsA37DS4EHMr0ISeZgpdl46o6Zvo0z0KsDugaQTgqKkB y7rIW3icaPbxi+2yrMrJ3bwqUSULg2UZqDj6fyzJvU+qKVSUOQ7S6xBJUmREzT4MmkLd8U9TeGPvv ymDq47f6DAtYNr58lVVZDq6TSRoskqFlNECCPGZEbNrLobvA37htwyN3Aehsa618U/sNT3Ql3LZvF TlHPhkJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wgk6S-0000000ChjD-2cg3; Mon, 06 Jul 2026 14:15:20 +0000 Received: from canpmsgout08.his.huawei.com ([113.46.200.223]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wgk6P-0000000ChiI-1cXv for linux-arm-kernel@lists.infradead.org; Mon, 06 Jul 2026 14:15:19 +0000 dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=LEJv5GyWiRmG2lSUuloOO2+UEipCg7jbRxfrhXUgk8s=; b=41tMsLXMaDSYZJPXIupOQyQvb/gpqtDKslYG4fVwdZxBpGOjnW1vcH8VebS+y9NQsh6CqKZcO uSjqIilj09n3cowX5IoRtYcC0hk5wlZzZF+Srsrb5YASwLtv4t1PXDZ9xFMyjYstbfK1DqrnKCM kFdeNgQVKTmIQ8P3fZ0lD8w= Received: from mail.maildlp.com (unknown [172.19.162.92]) by canpmsgout08.his.huawei.com (SkyGuard) with ESMTPS id 4gv5jr67xRzmV7V; Mon, 6 Jul 2026 22:05:52 +0800 (CST) Received: from dggemv705-chm.china.huawei.com (unknown [10.3.19.32]) by mail.maildlp.com (Postfix) with ESMTPS id F268540565; Mon, 6 Jul 2026 22:15:05 +0800 (CST) Received: from kwepemq500001.china.huawei.com (7.202.195.224) by dggemv705-chm.china.huawei.com (10.3.19.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 6 Jul 2026 22:15:05 +0800 Received: from [10.67.146.137] (10.67.146.137) by kwepemq500001.china.huawei.com (7.202.195.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 6 Jul 2026 22:15:04 +0800 Subject: Re: Question about the "TLBs and I-cache are private to each vCPU" guarantee with VTTBR_EL2.CnP To: Marc Zyngier CC: Wei-Lin Chang , , , , , , , , , , , "guoyang (C)" , "huanglingyan (A)" , "Wangzhou (B)" References: <292b5734-9005-6db0-da08-3da04628e620@huawei.com> <86o6gkpokm.wl-maz@kernel.org> <21eb51aa-443c-4d08-b4dd-3f813bbc9880@huawei.com> <86jyr8pkw8.wl-maz@kernel.org> From: Tangnianyao Message-ID: <5685cdb9-95d8-9ead-4d24-d6ad06dd9547@huawei.com> Date: Mon, 6 Jul 2026 22:15:04 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <86jyr8pkw8.wl-maz@kernel.org> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [10.67.146.137] X-ClientProxiedBy: kwepems100002.china.huawei.com (7.221.188.206) To kwepemq500001.china.huawei.com (7.202.195.224) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260706_071518_108603_68B7F43F X-CRM114-Status: GOOD ( 27.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 7/6/2026 16:44, Marc Zyngier wrote: > On Mon, 06 Jul 2026 09:25:46 +0100, > Tangnianyao wrote: >> >> >> On 7/6/2026 15:25, Marc Zyngier wrote: >>> On Mon, 06 Jul 2026 04:30:30 +0100, >>> Tangnianyao wrote: >>>> >>>> On 7/6/2026 1:28, Wei-Lin Chang wrote: >>>>> Hi, >>>>> >>>>> Let me try to answer this: >>>>> >>>>> On Sat, Jul 04, 2026 at 03:45:56PM +0800, Tangnianyao wrote: >>>>>> Hi, all >>>>>> >>>>>> I'm trying to understand the TLB and I-cache invalidation in >>>>>> `kvm_arch_vcpu_load()` that is intended to "guarantee that both TLBs and >>>>>> I-cache are private to each vCPU". >>>>>> >>>>>> As I understand it, when `VTTBR_EL2.CnP == 1`, `__kvm_flush_cpu_context()` >>>>>> only performs a local TLB and I-cache invalidation, which does not seem >>>>>> sufficient to guarantee that property. >>>>>> >>>>>> In fact, even if the invalidation were extended to the Inner Shareable >>>>>> domain, it still seems difficult to guarantee “TLBs and I-cache are >>>>>> private to each vCPU”, when `VTTBR_EL2.CnP == 1`, as long as multiple >>>>>> vCPUs from the same VM may be running concurrently on different PEs. >>>>> I think you have missed that when 2 stages are involved, both stages >>>>> have to set CnP == 1 in order to share TLB entries (Arm ARM R_ZVRZW). >>>>> So if TLB entry sharing happens, the guest kernel must have allowed it >>>>> in the first place (by setting TTBR0/1_EL1.CnP == 1), hence accidental >>>>> sharing that you are worried about won't happen. >>>>> >>>>> __kvm_flush_cpu_context() is solving problems that occur when multiple >>>>> vCPUs of a VM are multiplexed on a single physical CPU. >>>> Thanks for you answer. >>>> >>>> If guest kernel allow TLB shared across CPUs by setting TTBR0/1_EL1.CnP == 1, >>>> does kvm still need to guarantee that TLBs are private to each vCPU? >>> Yes, because there is nothing that describes which physical CPUs >>> actually share TLBs. So the only possible course of action is to >>> ignore what the guest says and fallback to something that is safe. >>> >>>>>> So I have two questions: >>>>>> >>>>>> 1. What is the rationale behind the comment that "guarantee that both TLBs >>>>>> and I-cache are private to each vCPU"? >>>>> I assume you are asking why keeping both TLBs and I-cache private per >>>>> each vCPU is required. The fundamental answer is that each physical CPU >>>>> is expected to have its own TLB and I-cache, so we must uphold that >>>>> property for vCPUs as well. vCPUs can be scheduled on the same physical >>>>> CPU, and use the same physical TLB/I-cache, obviously, so extra >>>>> invalidations need to be done. >>>> Let's assume that both Stage-1 CnP and Stage-2 CnP are enabled. >>>> >>>> As I understand it, the architecture permits TLB to be shared by multiple >>>> PEs within an Inner Shareable domain. Right? >>>> >>>> If an implementation allows TLB entries to be shared in this way, it seems >>>> that the current invalidation performed by kvm would no longer be sufficient >>>> to guarantee that TLBs are private to each vCPU. >>> Care to explain why? >>> >>> The core assumption is that a TLBI take effect on all the PEs the TLB >>> is shared with. If this doesn't work, then CnP is unusable, because it >>> is then impossible to guarantee that a translation will be refetched >>> (you could always hit in another PEs TLBs). Such an implementation >>> would be terminally broken. >>> >>> M. >>> >> For example: >> Sharing the TLB between the two SMT threads of the same physical core can >> reduce hardware cost while increasing the effective TLB coverage. > I have a precise idea of what TLB sharing can achieve. > >> A local TLBI take effect on the whole TLB shared by the two SMT threads, >> with the sharing enabled by CnP. >> >> In this scenario, enabling CnP in KVM appears to break the guarantee that >> TLBs are private to each vCPU when multiple vCPUs of the same VM run >> concurrently on different SMT threads of the same PE. > Please stop inventing your own terminology. A PE *is* a thread in the > architecture. There is no such thing as "threads of the same PE". > > But more to the point: if TLBI invalidates the relevant TLBs for all > the PEs that share them, *why* isn't the current KVM behaviour not > enough to ensure that the vcpu will not hit old TLBs that are there as > a result of a vcpu having run there previously? > > Please explain. > > M. > Two SMT threads(PE0,PE1) on the same physical core share TLB. VM0 has 2 vcpus, vcpu0 and vcpu1 that share all architectural context except the address translation context. Vcpu0 may observe TLB entries that are supposed to be private to vcpu1 in the following case: PE0(core0,smt0) PE1(core0,smt1) vcpu0 load vcpu0 va->pa0 vcpu0 put vcpu1 load vcpu1 flush local tlb vcpu1 modify desc to va->pa1 vcpu0 load vcpu0 hit *va->pa1* Thanks Nianyao Tang