From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0919C54EE9 for ; Tue, 6 Sep 2022 08:53:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB2038026F; Tue, 6 Sep 2022 04:53:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E612080224; Tue, 6 Sep 2022 04:53:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D50368026F; Tue, 6 Sep 2022 04:53:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C865080224 for ; Tue, 6 Sep 2022 04:53:29 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 90DE012072E for ; Tue, 6 Sep 2022 08:53:29 +0000 (UTC) X-FDA: 79881046938.21.4CD3B01 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf20.hostedemail.com (Postfix) with ESMTP id 22EC61C008C for ; Tue, 6 Sep 2022 08:53:27 +0000 (UTC) Received: from canpemm500009.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4MMJwq2Zl9zmVCj; Tue, 6 Sep 2022 16:49:47 +0800 (CST) Received: from [10.67.102.169] (10.67.102.169) by canpemm500009.china.huawei.com (7.192.105.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 6 Sep 2022 16:53:20 +0800 CC: , , , , , , , , , , , , , , , , Barry Song <21cnbao@gmail.com>, , , , Subject: Re: [PATCH v3 0/4] mm: arm64: bring up BATCHED_UNMAP_TLB_FLUSH From: Yicong Yang To: , , , , , , References: <20220822082120.8347-1-yangyicong@huawei.com> Message-ID: <34762b48-3da4-4f2e-64a7-68a44f21d4fd@huawei.com> Date: Tue, 6 Sep 2022 16:53:20 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: <20220822082120.8347-1-yangyicong@huawei.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.67.102.169] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500009.china.huawei.com (7.192.105.203) X-CFilter-Loop: Reflected ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662454408; a=rsa-sha256; cv=none; b=ljRI+ThQShuOVBJKhTA/u4+cECsbDvPbByCcZvSkVDbyoiekeuWbAcVR5fblTho+eBRmbi yZO5WiOeZ0M/peK+kz5O2+XAG+FVxxcsP61nxIPcqtdS55PJGmXgeblWVjzaCKuNt8A38k maJTQY2x0Bpkeuw/IIZaFTQFCHpeBGs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=yangyicong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662454408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LEdyqWBwfz2fDAnbNhvMFdTL091pdlSeqLYqsZmVKRs=; b=Fmw9MTSyCfp+b9Kn2MFwzuguWxjCvlqH3nwnlAfkLsyW9/Nxc3yqqUz23GgeClGVE24jk/ dSCqhWvkTqUxsBmvH03PyQLI/jfvChiMusbxVDE8SJxxkZ+dpPhRQnTN5mYZJhMeRWshfb N9H0Ra04wrAY2wMp3fDYRjcSYzqyaZI= X-Rspamd-Queue-Id: 22EC61C008C Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=yangyicong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: gbnyjdf9zjupro485sipxi4na18t9qaz X-HE-Tag: 1662454407-753527 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi mm and arm64 maintainers, a gentle ping for this.. Thanks. On 2022/8/22 16:21, Yicong Yang wrote: > From: Yicong Yang > > Though ARM64 has the hardware to do tlb shootdown, the hardware > broadcasting is not free. > A simplest micro benchmark shows even on snapdragon 888 with only > 8 cores, the overhead for ptep_clear_flush is huge even for paging > out one page mapped by only one process: > 5.36% a.out [kernel.kallsyms] [k] ptep_clear_flush > > While pages are mapped by multiple processes or HW has more CPUs, > the cost should become even higher due to the bad scalability of > tlb shootdown. > > The same benchmark can result in 16.99% CPU consumption on ARM64 > server with around 100 cores according to Yicong's test on patch > 4/4. > > This patchset leverages the existing BATCHED_UNMAP_TLB_FLUSH by > 1. only send tlbi instructions in the first stage - > arch_tlbbatch_add_mm() > 2. wait for the completion of tlbi by dsb while doing tlbbatch > sync in arch_tlbbatch_flush() > My testing on snapdragon shows the overhead of ptep_clear_flush > is removed by the patchset. The micro benchmark becomes 5% faster > even for one page mapped by single process on snapdragon 888. > > -v3: > 1. Declare arch's tlbbatch defer support by arch_tlbbatch_should_defer() instead > of ARCH_HAS_MM_CPUMASK, per Barry and Kefeng > 2. Add Tested-by from Xin Hao > Link: https://lore.kernel.org/linux-mm/20220711034615.482895-1-21cnbao@gmail.com/ > > -v2: > 1. Collected Yicong's test result on kunpeng920 ARM64 server; > 2. Removed the redundant vma parameter in arch_tlbbatch_add_mm() > according to the comments of Peter Zijlstra and Dave Hansen > 3. Added ARCH_HAS_MM_CPUMASK rather than checking if mm_cpumask > is empty according to the comments of Nadav Amit > > Thanks, Peter, Dave and Nadav for your testing or reviewing > , and comments. > > -v1: > https://lore.kernel.org/lkml/20220707125242.425242-1-21cnbao@gmail.com/ > > Anshuman Khandual (1): > mm/tlbbatch: Introduce arch_tlbbatch_should_defer() > > Barry Song (3): > Revert "Documentation/features: mark BATCHED_UNMAP_TLB_FLUSH doesn't > apply to ARM64" > mm: rmap: Extend tlbbatch APIs to fit new platforms > arm64: support batched/deferred tlb shootdown during page reclamation > > Documentation/features/arch-support.txt | 1 - > .../features/vm/TLB/arch-support.txt | 2 +- > arch/arm64/Kconfig | 1 + > arch/arm64/include/asm/tlbbatch.h | 12 ++++++++ > arch/arm64/include/asm/tlbflush.h | 28 +++++++++++++++++-- > arch/x86/include/asm/tlbflush.h | 15 +++++++++- > mm/rmap.c | 19 +++++-------- > 7 files changed, 61 insertions(+), 17 deletions(-) > create mode 100644 arch/arm64/include/asm/tlbbatch.h >