From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78D5DC83F22 for ; Tue, 15 Jul 2025 09:34:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 912366B00AD; Tue, 15 Jul 2025 05:34:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89BA96B00AE; Tue, 15 Jul 2025 05:34:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7638B6B00AF; Tue, 15 Jul 2025 05:34:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5F2A26B00AD for ; Tue, 15 Jul 2025 05:34:29 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 29D5EB8AED for ; Tue, 15 Jul 2025 09:34:29 +0000 (UTC) X-FDA: 83665988658.04.D2B07B7 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf20.hostedemail.com (Postfix) with ESMTP id 449211C000C for ; Tue, 15 Jul 2025 09:34:27 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QX+sKinm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3oSB2aAUKCAYzghhgmuumrk.iusrot03-ssq1giq.uxm@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3oSB2aAUKCAYzghhgmuumrk.iusrot03-ssq1giq.uxm@flex--tabba.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752572067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FLr2IgfKpLCEHT7MXIG6C0hRgCJ9BMJYV3UvbtKXymU=; b=fGQPEv5/u68kKZFCxHBsj/HTei7MyeWJBySaMdSww/GgV0brg8bAo3Dk+iOY5nEBzymXKp j7mB/8mCm1bc5SLHrTllGo3VGULUI3FgQIe9VVqI70tfq4LGE3qXp1XICmeJDk8s4z31Vx 0zGcPDS6lWpTmp1HXRLZbpdpeLDfXQs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752572067; a=rsa-sha256; cv=none; b=Yvg5+oTlClyt7DzJykVwDvb9aAq/pU9RA301Ea+zsHSeN1edWrh8Rt03invMvZcHpItB9y 0PjVY0N2dEKJT5cMlFBoTISl312KxyAA5aYEPKJVuZ1m6pEI1pioqA3bZDfdFBT4i+H37R A/e8FN4dyjSOzPjyqboMsl+3dbGiqJE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QX+sKinm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3oSB2aAUKCAYzghhgmuumrk.iusrot03-ssq1giq.uxm@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3oSB2aAUKCAYzghhgmuumrk.iusrot03-ssq1giq.uxm@flex--tabba.bounces.google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4538a2f4212so30124935e9.2 for ; Tue, 15 Jul 2025 02:34:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752572066; x=1753176866; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FLr2IgfKpLCEHT7MXIG6C0hRgCJ9BMJYV3UvbtKXymU=; b=QX+sKinmCAPE8l5hmunidJLwmB5W2E4RGbVsZobXRxwULhL/LdRH2j73K6C/9yKmEm TK30ZEQ71kjAqyZw5af+Sd5ue0Z5bBtX90+ru5JwHkKwXTBOb19cJesuDffj3zTKGa0T yof1CksVickPxJvGh/9Ku38/ZPqRZdvdTclpHCkIjBB+dq/SGjGSGBBU87uRfkMMwwxB KyilyznzvdIOHtfCTzG7Y3msfdZerSlW8uV5XaWWSyziyQP371ezVE79nMrvugTy64IL 0GHObbapJQhLzLHBS/oUxuy7cDytdWUFiUcE52rNL/qiIqegnUtTbSYuZS08gbvNLrWu opDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752572066; x=1753176866; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FLr2IgfKpLCEHT7MXIG6C0hRgCJ9BMJYV3UvbtKXymU=; b=lL89r1GQHXhxcNjRSgXe9nM06i/KuswHNYoGMDTEc8f2y2fC0vI0aMUW2F13NGQ+sA 4xOXb3bJIQcEU1tnunPkbpRBp5cI1bszYyd1ErOQ4W0IIiMGxLxDrRpAyr/s5TKEZiEZ mmc3TdHI3lMaRZ2ULvipWSe7/Ee8/gZikeT5hvRIBEqxx+l2xOQoyzMtiOnSgPM6whYG MiVE64WbRYG++J0L5A1xxvv9/ehA/0qF84plVWHfP2PubZs+KTr5U2hV6n76jQDLYdQb esPSWkTNnkKMc46mjT1Ntc/KhymSEBqLGBzcd0Jx/v3971X/kmgjU5MQOxGxDBzgZAUN 2JGg== X-Forwarded-Encrypted: i=1; AJvYcCU1WlWeGIQ9hODJJSHK0JVa6GIHRHHxSwhRE9+Jq1iAcER/p4xGPdlGQwhOQOR11w7LIpvp3D2ctQ==@kvack.org X-Gm-Message-State: AOJu0YxTjb+fjft4PLcAE316U11XZY184/YgTjik0HNhm93txYSPSdck XbrBB0Nf7nBcmX/YkV9+32Mh5F5q4GSdXzW0m+naph8uTPeO8lUjjNYISC+/lGnf9rXZ91Bgj0z xyg== X-Google-Smtp-Source: AGHT+IGr/ToTExaXd1fztA3oBZSTLF4qbi6m2OY0gu5O+xOxmi0/NqRRIq7T+fjwKX6tdN/jPkynWyRVuA== X-Received: from wmsd13.prod.google.com ([2002:a05:600c:3acd:b0:453:7edf:24ff]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a216:b0:450:d386:1afb with SMTP id 5b1f17b1804b1-45555e4c191mr103954565e9.9.1752572065745; Tue, 15 Jul 2025 02:34:25 -0700 (PDT) Date: Tue, 15 Jul 2025 10:33:45 +0100 In-Reply-To: <20250715093350.2584932-1-tabba@google.com> Mime-Version: 1.0 References: <20250715093350.2584932-1-tabba@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250715093350.2584932-17-tabba@google.com> Subject: [PATCH v14 16/21] KVM: arm64: Handle guest_memfd-backed guest page faults From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org, kvmarm@lists.linux.dev Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, pankaj.gupta@amd.com, ira.weiny@intel.com, tabba@google.com Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: fgwm1bjon9hzmguwpjojgc5crrbei5g1 X-Rspamd-Queue-Id: 449211C000C X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1752572067-609922 X-HE-Meta: U2FsdGVkX19wjzYemKcwK75jDf21hXXIpOsBdF7amfWYFZ31ZAWMzm5CX+68G7QiXa+rHVXi2GV0QKX09bJoFc7MV8rqv1HG3R4JVxBl2/eXtt1Mh7nWwAeRmsQ87+wfHXOyFyWwqln8ook12Bzid+2+CqYhcYUUFi65yQhkpxMv8olPA0FDnOrvb1lns4QFdPajEcHuMZ4wZE17JZZo+J8MJteKLTxv7adh8Nt1xNtdEPE9hXwkgv6iaprU1HYKpJFTV08dx/LS8UywySAIAp/xmMYAyecM/eyAE8RJ1VYCeLxJNjEZpQZr+zfqKtLYnBrN+K6jkqV4WZqQAvlMx7EO0H0xWXBVKzrQkUq/oMgb+bFNTuPa/14jkDitZ/M2xhvJva14W2Rq89mUNKzqM5Qq+RqHhKcr9Yu90j5tQgAUVrslSri6UXHQpyCU8pf+1oQH8m4gqPrm31YC5UG23pIEYrDSVuW7YcUsqOFxRr3AGfT3L+ZLRAmC+Q8GqtNd1BEOGoNe4yuY1p/bctgz2c/nH+97Y24bd8YApxL88ueQtYYPvasLop12fmoPMZTS7zIjqpAs0wjBqnkVlQ6kfYAS+7h+7ebj0icTMQe6itpGI9IFFOXx0trlyG4bTHcT9IQCNflHtmJdsVQQpmg/A65EdZE4M/dlX7EfijuHkVu2tBORj+n52K4hp0OyN/UpH+XLLd9OUu6iKPM7Wg0zhcVPOQGwR75oLKxJhmisxg44bluVeOqrw7YAdEtv82sAZR98H8OCMVMFg4nPoctQDlnusFrGPKoHG/bEOcciq22+H4eZQ+t7l/RKhxSsN6hmLXU+2NZsPi7X+jWV4EfWc85N7YQ4KABUyVPrEasj4oYGgLsUwOmkhOcPf07WRqe1TwcOOGThbYAaxB/qUsCGn10WNt1fO7YJXubUuBoJSohbcE0cnoiOBwG3XSt6gGIUYvFzCgcTgkLdRpHzoYW xudqCRe7 fq7SnbTNVjHIBbdg4+rTTW2xogQkas/R/ap9fRjxnLm0QIGZsSWVnmWIr1lhBYRWZ/rwWa/x2UI5YGCQbwlq+vNC33+6Bs4mXOf0K36BieqcJ8QhULJG7Jhnca3StKjQj+wU7LfsRiaCTiCGXJN0m0MfGIkUW3bHmjyvDSMsqugeTZTYyXTgVKklzUcg+hXoq6Wd9SmpRR1P3hO3yN1IatHZI4p+LWpp09S731YyOsaScNnXAxETR6JDhPl5ooQO/fNotAFQufplDSk5pml7vkY76oHgjmZBRIXdXElwvTXeO3oy8OVBcWbmyDSq9jD6GyAu67Yvbc00pvdJpuMBUWWeNksBU+4O8dm7UdhnfDTQNkeuighpiAfUgmYczUuy84+EWm2WkAcsV9uq4EKUsVtrZnBfpBS4AYOY77heycA3EB6upvMM9p4tWABoX9jyFSHe1UxK1HRTBmBHdj/f1Unu4oipU6H0rI307x9MFYe1fk+6A0CLDawQOVCB8oVTbnuZY5OWDP48XyY94pZLjmOCAfmSmOXrhSlvnOhxzVrcp18xKqvE+J3wbhyYW3viG4p94BL3MqwUj0pyNLGKO0XQ3YVBkNdN0XlosmN3B7l2myEBpR5ufOXmEECsZ6PW1j68cntg3sq57HeUqwx/wP23tpIRoACI2Sarvs61OkBPHyag= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add arm64 architecture support for handling guest page faults on memory slots backed by guest_memfd. This change introduces a new function, gmem_abort(), which encapsulates the fault handling logic specific to guest_memfd-backed memory. The kvm_handle_guest_abort() entry point is updated to dispatch to gmem_abort() when a fault occurs on a guest_memfd-backed memory slot (as determined by kvm_slot_has_gmem()). Until guest_memfd gains support for huge pages, the fault granule for these memory regions is restricted to PAGE_SIZE. Reviewed-by: Gavin Shan Reviewed-by: James Houghton Signed-off-by: Fuad Tabba --- arch/arm64/kvm/mmu.c | 86 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 83 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index b3eacb400fab..8c82df80a835 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1512,6 +1512,82 @@ static void adjust_nested_fault_perms(struct kvm_s2_trans *nested, *prot |= kvm_encode_nested_level(nested); } +#define KVM_PGTABLE_WALK_MEMABORT_FLAGS (KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED) + +static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, + struct kvm_s2_trans *nested, + struct kvm_memory_slot *memslot, bool is_perm) +{ + bool write_fault, exec_fault, writable; + enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS; + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; + struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt; + unsigned long mmu_seq; + struct page *page; + struct kvm *kvm = vcpu->kvm; + void *memcache; + kvm_pfn_t pfn; + gfn_t gfn; + int ret; + + ret = prepare_mmu_memcache(vcpu, true, &memcache); + if (ret) + return ret; + + if (nested) + gfn = kvm_s2_trans_output(nested) >> PAGE_SHIFT; + else + gfn = fault_ipa >> PAGE_SHIFT; + + write_fault = kvm_is_write_fault(vcpu); + exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu); + + VM_WARN_ON_ONCE(write_fault && exec_fault); + + mmu_seq = kvm->mmu_invalidate_seq; + /* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */ + smp_rmb(); + + ret = kvm_gmem_get_pfn(kvm, memslot, gfn, &pfn, &page, NULL); + if (ret) { + kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE, + write_fault, exec_fault, false); + return ret; + } + + writable = !(memslot->flags & KVM_MEM_READONLY); + + if (nested) + adjust_nested_fault_perms(nested, &prot, &writable); + + if (writable) + prot |= KVM_PGTABLE_PROT_W; + + if (exec_fault || + (cpus_have_final_cap(ARM64_HAS_CACHE_DIC) && + (!nested || kvm_s2_trans_executable(nested)))) + prot |= KVM_PGTABLE_PROT_X; + + kvm_fault_lock(kvm); + if (mmu_invalidate_retry(kvm, mmu_seq)) { + ret = -EAGAIN; + goto out_unlock; + } + + ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, PAGE_SIZE, + __pfn_to_phys(pfn), prot, + memcache, flags); + +out_unlock: + kvm_release_faultin_page(kvm, page, !!ret, writable); + kvm_fault_unlock(kvm); + + if (writable && !ret) + mark_page_dirty_in_slot(kvm, memslot, gfn); + + return ret != -EAGAIN ? ret : 0; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_s2_trans *nested, struct kvm_memory_slot *memslot, unsigned long hva, @@ -1536,7 +1612,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; struct page *page; - enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED; + enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS; if (fault_is_perm) fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1961,8 +2037,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) && !write_fault && !kvm_vcpu_trap_is_exec_fault(vcpu)); - ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, - esr_fsc_is_permission_fault(esr)); + if (kvm_slot_has_gmem(memslot)) + ret = gmem_abort(vcpu, fault_ipa, nested, memslot, + esr_fsc_is_permission_fault(esr)); + else + ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, + esr_fsc_is_permission_fault(esr)); if (ret == 0) ret = 1; out: -- 2.50.0.727.gbf7dc18ff4-goog