From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9FB8C83F14 for ; Wed, 30 Aug 2023 21:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Dej2xtDBc1zVEj1xRAt3olSE3zwVdwQuKpycBRkGUzY=; b=2+EN2cfHLLUjOX 4a4xUOfmvdy4GrLY1LiJdbjamzl1SYFEZXch0xNRJerhN7kykYPnjiUP7Qg5PT4zhypVMr8e7xYxW zdp17ncBTTer+e8tJbNd3MozZPHRKF9kLlbhjLX+PPIkV+5Q6FAZzYpHbVepE3YZmiIctUU2OrfOg 4+tkj+uBxOgZv+6St/0j7yh45keV0x+Ce61zfD6IkPXVRK1anV2K1e6/oedlF2a2TTOOKFLU5fsu0 7QaPkFb0AaLhTCKSaS61ruarXksQa4PmpPRDM3nabDAWuoZ4klkidLWB8695dEVC+EuPejvYmuF0n ok1tBp/5E76rSzWREphw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qbT4l-00EE20-0W; Wed, 30 Aug 2023 21:50:11 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qbT4h-00EE0z-33 for linux-arm-kernel@lists.infradead.org; Wed, 30 Aug 2023 21:50:09 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CA070B82353; Wed, 30 Aug 2023 21:50:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C09E1C433C7; Wed, 30 Aug 2023 21:50:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1693432204; bh=OGc+LfkYBQySHlSntBlsKXI/UcufIS548SjZfr7mroQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Rd8Q6DhBwUgRClZMRUKSEtwN+6EDSNQNpVA6FiKuJBnQqyJpF6gZFIb8quGIcb98H iTP73712DKtlzoti6gemSAePS8lw2/HXExBVPWLM1vOAeykoKCr8kSskeK9uWCI9AJ GhjTiXJPvmx6hCAHk4SWdAX2tVVD3JBBozMWBQ9PKq+q2F6EKbDDywiKytQgMO9DNa KDeei3iDgN4iSl3oRaHB4PzRi6Xb9UpO39ErUtl9DETNSQ/GSpU14Ojk9PSOtN3AkY O8H3Bi2n4ymg08fd3dTTh6qdcoQzm2kvv2e8aH4O12IlVIiL7ssD10FLC5rdfa2PWd yysHdw5ehQ8Sg== Date: Wed, 30 Aug 2023 22:49:59 +0100 From: Will Deacon To: Nicolin Chen Cc: Robin Murphy , jgg@nvidia.com, joro@8bytes.org, jean-philippe@linaro.org, apopple@nvidia.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev Subject: Re: [PATCH 1/3] iommu/io-pgtable-arm: Add nents_per_pgtable in struct io_pgtable_cfg Message-ID: <20230830214958.GA30121@willie-the-truck> References: <0fe68babdb3a07adf024ed471fead4e3eb7e703f.1692693557.git.nicolinc@nvidia.com> <61f9b371-7c45-26b1-ec0f-600765280c89@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230830_145008_313022_2F8F8B02 X-CRM114-Status: GOOD ( 21.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 29, 2023 at 03:15:52PM -0700, Nicolin Chen wrote: > Meanwhile, by re-looking at Will's commit log: > arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE > > In order to reduce the possibility of soft lock-ups, we bound the > maximum number of TLBI operations performed by a single call to > flush_tlb_range() to an arbitrary constant of 1024. > > Whilst this does the job of avoiding lock-ups, we can actually be a bit > smarter by defining this as PTRS_PER_PTE. Due to the structure of our > page tables, using PTRS_PER_PTE means that an outer loop calling > flush_tlb_range() for entire table entries will end up performing just a > single TLBI operation for each entry. As an example, mremap()ing a 1GB > range mapped using 4k pages now requires only 512 TLBI operations when > moving the page tables as opposed to 262144 operations (512*512) when > using the current threshold of 1024. > > I found that I am actually not quite getting the calculation at the > end for the comparison between 512 and 262144. > > For a 4K pgsize setup, MAX_TLBI_OPS is set to 512, calculated from > 4096 / 8. Then, any VA range >= 2MB will trigger a flush_tlb_all(). > By setting the threshold to 1024, the 2MB size bumps up to 4MB, i.e. > the condition becomes range >= 4MB. > > So, it seems to me that requesting a 1GB invalidation will trigger > a flush_tlb_all() in either case of having a 2MB or a 4MB threshold? > > I can get that the 262144 is the number of pages in a 1GB size, so > the number of per-page invalidations will be 262144 operations if > there was no threshold to replace with a full-as invalidation. Yet, > that wasn't the case since we had a 4MB threshold with an arbitrary > 1024 for MAX_TLBI_OPS? I think this is because you can't always batch up the entire range as you'd like due to things like locking concerns. For example, move_page_tables() can end up invalidating 2MiB at a time, which is too low to trigger the old threshold and so you end up doing ever single pte individually. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel