From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A281C43458 for ; Mon, 13 Jul 2026 10:09:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KjLkeM3nlsHuqO4OxO/eUTJdfCtxEk0PbX1Q4tFQhdY=; b=X0TYG5LeemqsCcrLqTQxjsAjpO +3jDiKAL2cfZY/HTxPqByu4GQVdOAfXUt+3gZIRJNGUQUdT9kNwOY/pmhayaQCOTH13ffIR+HFPVU 2GPVYzxRtnVV9r1QrbDj0wv5kjuJl5NVfs5xhRXPKl9AFHa1tDHOX9AOiAZbALLygMH0YahEtvyo0 iqsYjnF5Nc3L/d/SxEOq6TjR2jzWiuWufXG7HGWiOpw4EbZ8oFG2P02fH5L7nCAe9P1hglJDXC3ek GYpu+vFSQ21nwTgalessWM9pIDx1CVdGNg7SpEpRSB4veRkdds0TxuRgvkeHJDFFedABSOCeORipC 6vidaEbQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wjDan-00000008pmn-2xbb; Mon, 13 Jul 2026 10:08:53 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wjDak-00000008plg-0a3f for linux-arm-kernel@lists.infradead.org; Mon, 13 Jul 2026 10:08:51 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 92D631576; Mon, 13 Jul 2026 03:08:42 -0700 (PDT) Received: from LeoBrasDK.cambridge.arm.com (LeoBrasDK.cambridge.arm.com [10.2.212.21]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B5F9A3F93E; Mon, 13 Jul 2026 03:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1783937326; bh=iGYWauEyEI7qdxoh+eeLhq0GerGnoujG8/HnZ2A/9b4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PpMeKrdNP+xzor49EHWl9BEjBCpAFz8tKR8/wQlOKuPv/BvWV8vEJXg1qaOhKKzJK R71I7rt+UW6LUp5+/lxEY2RoFL8TycAL/5RSYWFOODNkKgHLkLWX0p1JUOrpLU76sy TOpGX/ATRzH/2mDnVYu7DP/LLJvMsjQkS63lsFjs= From: Leonardo Bras To: Dev Jain Cc: Leonardo Bras , Marc Zyngier , Oliver Upton , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Fuad Tabba , Raghavendra Rao Ananta , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 0/3] Optimize S2 page splitting Date: Mon, 13 Jul 2026 11:08:38 +0100 Message-ID: X-Mailer: git-send-email 2.55.0 In-Reply-To: <3cb5f1d7-f1cb-44bc-925c-85b5b6909b64@arm.com> References: <20260708134101.2514759-1-leo.bras@arm.com> <3cb5f1d7-f1cb-44bc-925c-85b5b6909b64@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260713_030850_336750_253ABB82 X-CRM114-Status: GOOD ( 33.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sun, Jul 12, 2026 at 08:20:28PM +0530, Dev Jain wrote: > > > On 08/07/26 7:10 pm, Leonardo Bras wrote: > > While playing with dirty-bit tracking, I decided to take a look on how page > > splitting works. Found out all entries are walked, even though we can infer, > > for instance that: > > - If a level-3 entry is walked, it means the parent level-2 entry is split > > - If a split just succeeded in an table entry, it means all children nodes > > are already split Hi Dev, thanks for reviewing! > > That does not happen for force_pte = false right? Or is it guaranteed that > in your case that force_pte = true always? IIUC for kvm_pgtable_stage2_split(), force_pte will affect the creation of pagetables, and will only happen to be false if the cache used for splitting hugepages does not contain enough pages to split the whole subtree. In this case, it only maps one page of either table entries or ptes. I am not sure how often does that happen for the memory cache not being able to split everythig, but I guess it's only when memory is tight in the system. (Haven't checked, I can be wrong here) But in any case, the idea of optimizing this was discarted because it ended up being faster to just skip by level. > > > > This patches' idea is to introduce new walking flags to skip pagetable > > levels 0-3. > > > > The idea of skipping child nodes was also tested, but it was marginally > > slower than just skipping levels, so it was discarted. ^ Here Thanks! Leo > > > > Optimization measured on two scenarios involving eager-splitting on a > > VM with 32 memslot of 2GB (total 64GB), and vcpu per slot: > > - Scenario 1: No manual protect, whole memslot split at dirty-track enable > > (KVM_SET_USER_MEMORY_REGION2 ioctl with KVM_MEM_LOG_DIRTY_PAGES) > > - Split happens only once, whole region > > - Evalutes improved batch performance of splitting > > - Scenario 2: Manual protect, split happens during every dirty-bit clean > > (KVM_CLEAR_DIRTY_LOG ioctl), average for 2 iterations. > > - Split called multiple times, for smaller 64-page sections. > > - Evaluate improved performance for multiple calls > > > > Scenario 1, improvement on dirty-track enable ioctl for the memslot: > > - Memory was already split (4k pages): -43.18% runtime (stdev 3.87%) > > - THP backed memory: -25.77% runtime (stdev 1.01%) > > - 64x1GB hugetlb memory: -25.62% runtime (stdev 1.09%) > > > > Scenario 2, improvement on dirty-log clean ioctl for the memslot: > > - Memory was already split (4k pages): -39.88% runtime (stdev 2.71%) > > - THP backed memory: -25.18% runtime (stdev 0.32%) > > - 64x1GB hugetlb memory: -49.17% runtime (stdev 1.86%) > > > > For collecting above numbers, the following script was ran in both vanilla > > and patched kernels, with kernel parameter 'default_hugepagesz=1G', on an > > TX2 with 128GB RAM. > > > > --- dirty_test.sh > > #!/bin/bash > > filename=$(uname -r |cut -d'-' -f 4-) > > > > run_test(){ > > base_test="./dirty_log_perf_test -b 2G -v 32 -m 6 -m 8" > > > > # Manual cleaning disable > > ${base_test} -g > > ${base_test} -g -s anonymous_thp > > echo 64 > /proc/sys/vm/nr_hugepages > > ${base_test} -g -s shared_hugetlb > > echo 0 > /proc/sys/vm/nr_hugepages > > > > # Manual cleaning enable > > ${base_test} > > ${base_test} -s anonymous_thp > > echo 64 > /proc/sys/vm/nr_hugepages > > ${base_test} -s shared_hugetlb > > echo 0 > /proc/sys/vm/nr_hugepages > > } > > > > run_test 2>&1 | tee ${filename} > > --- > > > > Above dirty_log_perf_test command is the standard kvm selftest found in the > > kernel tree. It tested the following guest modes: > > Testing guest mode: PA-bits:40, VA-bits:48, 4K pages > > Testing guest mode: PA-bits:40, VA-bits:48, 64K pages > > (Modes with PA-bits:36 were discarted in this version, given the amount > > of RAM being used for testing, and the similarity of previous results) > > > > Performance numbers from above modes were used to calculate average and > > stdev showed in the optimization results. > > > > Changes since v2: > > - Rebased on top of v7.2-rc1 > > - Improved testing, added more memory, re-tested > > - Now: 32 vcpus @ total of 64G > > - Before: 1cpu @ 16G > > > > Changes since v1: > > - Fixed inverted flag verification priority (Sashiko) > > - Fixed incorrectly skipping POST call if level was skipped (Sashiko), and to that > > - New pre-patch that changes goto-out -> return to avoid re-testing walk_continue > > v1 Link: https://lore.kernel.org/lkml/20260610202112.2695205-2-leo.bras@arm.com/ > > > > Changes since RFC: > > - Changed approach from return value to walk flags (Will Deacon) > > - Discarted skip_child approach (Oliver Upton) > > - Measured in real hardware, and from userspace perspective (Marc Zyngier) > > - Better explanation of what and how numbers were collected > > RFC Link: https://lore.kernel.org/all/20260515195904.2466381-1-leo.bras@arm.com/ > > > > Thanks! > > Leo > > > > > > Leonardo Bras (3): > > KVM: arm64: Avoid re-testing walk_continue > > KVM: arm64: Introduce KVM_PGTABLE_WALK_SKIP_LEVEL* walk flags > > KVM: arm64: Make stage2_split_walker() skip unnecessary walks > > > > arch/arm64/include/asm/kvm_pgtable.h | 13 +++++++++++++ > > arch/arm64/kvm/hyp/pgtable.c | 28 +++++++++++++++++++++------- > > 2 files changed, 34 insertions(+), 7 deletions(-) > > > > > > base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482 >