From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37A3FC77B75 for ; Fri, 5 May 2023 12:29:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:To:From:References:Cc:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/zsbdRYKVej5aEiD7lVDTS7qmY6TP/FcLIgWW417KmE=; b=GV25VtmL3VsigO dYAeuAd4n/ig9HbJheaj8BbiUDVwW6yO1lxfFpwUAZKWE6M5WJBscNoO1jWHnYZ2FT2OL8BOURRjD ZoBCB/7pGzf2TekJS75KK9AYHTuBC/XRoRpxCYciFzHCe2RkihnOGMcia5TkQQLnb4Oy8jT49Ix6V TsJ+hoEmc0VyD9yunItQfszRheHFrOqKP7vGhIG0g6azsoe25rnUOCBHEJVnEE3oHrhZEFyvPxx/k +pAuOTS8ItDg5PRHzdbXUv0/bdYfggstccsx8AYy+/pl+1419156Jhu1k1so0HL6hyIxhtjGqXrOB 8VFf7f6qW5f22O2pDLVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1puuYL-00Asf3-1h; Fri, 05 May 2023 12:28:49 +0000 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1puuYI-00AseL-1Z for linux-arm-kernel@lists.infradead.org; Fri, 05 May 2023 12:28:48 +0000 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1aae5c2423dso16273795ad.3 for ; Fri, 05 May 2023 05:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1683289722; x=1685881722; h=content-transfer-encoding:in-reply-to:to:from:references:cc :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=ByJNnXSDRlq4gYdZRhAq3Xelfx/hEVGCr+0oMdNmBnI=; b=DMdHzFAvgx+VnA44zfWzCcq4X4hotlnfhegf8lM8tTPYFyZ1bUNUNjtjbPkVOIM5vb 8jKAm8xshSqZwZl5xuMDnliuDTYDewOHZ2zHqPUNYv8sdWc6mnSYfovi4VZMwh/zHI3B 66CY8T5ve6MCj70+Au+M0ymHLbZ7FG7lVBoaOIu8NSTJmxg80sfht7TOMxl94fjTB0cg ncIOTUASX31KN65+PUWp5lj1yHEB9IhJn5AVnPGzzNt2eg4BkJxLFVjgDe6d/O7mJxbW Q/B/Z/OHCfb81cSFds4NuiyRCfovw0w5bdwXbttZXKXMSiytz7CtjpKTYoDzqZlDATXq VdZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683289722; x=1685881722; h=content-transfer-encoding:in-reply-to:to:from:references:cc :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ByJNnXSDRlq4gYdZRhAq3Xelfx/hEVGCr+0oMdNmBnI=; b=CW497BCyoVfKz6QIO6tMjmRIrDu9N7VStUVgX7Yv0IrCQlRIIBILbgTU1RY8cPKydO vUZG56QFgRQelbsyDFM3ogEmgVKnVRIRVFtT7ZL0sdORNaArZ+8yTBJbPOLIwFfgb91G w38uy40/oLXMPUN0eQtqAg8eOrc79L/E8xcsWpHVaS/JJkfaqhxFajHSGQD7vbiQjToa qH1c+4RSmlXtLmGgWhOjPpf/iCsvyQ2dis/Vn+XymtDyioH658eHolTF/+4mnlSLtQlF PdkoiNeh22IYypwV1Wb7LoQ2Ff2YkN+6jwxuWuZxwzujcsz3wFG3MdgDLB9vYT9T5Ao3 B2PQ== X-Gm-Message-State: AC+VfDy3yUnCpl/r6fNYNdu+Zs4cOGA6Ls11bPvPSeq2Wsf4sFLNd5n4 Bk1zV2tkLtjaM9pM8vWvkKeE3w== X-Google-Smtp-Source: ACHHUZ4FrAb92tjqO0ADnTNF5JsiyFdayX/NIZ5p8gO0l1i/VTai1+/XNgLbTE+AUc4JW8p9H4Gl4Q== X-Received: by 2002:a17:902:a589:b0:1a6:d8a3:3346 with SMTP id az9-20020a170902a58900b001a6d8a33346mr1130936plb.31.1683289721733; Fri, 05 May 2023 05:28:41 -0700 (PDT) Received: from [10.2.117.253] ([61.213.176.10]) by smtp.gmail.com with ESMTPSA id u11-20020a17090282cb00b001a1ccb37847sm1669244plz.146.2023.05.05.05.28.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 05 May 2023 05:28:41 -0700 (PDT) Message-ID: <8d8c2ed5-c29b-8ea6-84b3-3335d0682d0d@bytedance.com> Date: Fri, 5 May 2023 20:28:35 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.1 Subject: Re: [QUESTION FOR ARM64 TLB] performance issue and implementation difference of TLB flush Content-Language: en-US Cc: Will Deacon , Tomasz Nowicki , Laura Abbott , Catalin Marinas , Ard Biesheuvel , Anshuman Khandual , Kefeng Wang , Feiyang Chen , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, x86@kernel.org References: <2eb026b8-9e13-2b60-9e14-06417b142ac9@bytedance.com> <369d1be2-d418-1bfb-bfc2-b25e4e542d76@bytedance.com> From: Gang Li To: Thomas Gleixner In-Reply-To: <369d1be2-d418-1bfb-bfc2-b25e4e542d76@bytedance.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230505_052846_731828_74D5FE36 X-CRM114-Status: GOOD ( 13.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, I found that in `ghes_unmap` protected by spinlock, arm64 and x86 have different strategies for flushing tlb. # arm64 call trace: ``` holding a spin lock ghes_unmap clear_fixmap __set_fixmap flush_tlb_kernel_range ``` # x86 call trace: ``` holding a spin lock ghes_unmap clear_fixmap __set_fixmap mmu.set_fixmap native_set_fixmap __native_set_fixmap set_pte_vaddr set_pte_vaddr_p4d __set_pte_vaddr flush_tlb_one_kernel ``` As we can see, ghes_unmap in arm64 eventually calls flush_tlb_kernel_range to broadcast TLB invalidation. However, on x86, ghes_unmap calls flush_tlb_one_kernel. Why arm64 needs to broadcast TLB invalidation in ghes_unmap, while only one CPU has accessed this memory area? Mark Rutland said in https://lore.kernel.org/lkml/369d1be2-d418-1bfb-bfc2-b25e4e542d76@bytedance.com/ > The architecture (arm64) allows a CPU to allocate TLB entries at any time for any > reason, for any valid translation table entries reachable from the > root in > TTBR{0,1}_ELx. That can be due to speculation, prefetching, and/or other > reasons. > > Due to that, it doesn't matter whether or not a CPU explicitly accesses a > memory location -- TLB entries can be allocated regardless. > Consequently, the > spinlock doesn't make any difference. > arm64 broadcast TLB invalidation in ghes_unmap, because TLB entry can be allocated regardless of whether the CPU explicitly accesses memory. Why doesn't x86 broadcast TLB invalidation in ghes_unmap? Is there any difference between x86 and arm64 in TLB allocation and invalidation strategy? Thanks, Gang Li _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel