From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5338C43458 for ; Tue, 14 Jul 2026 07:39:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:CC:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/ynr5BYYuvAAGgFG1JBPJ8ExJe4Z1HS483Kq6s4O+1k=; b=O/fMvDXDOMNvIVHgcJxiIC1gEs LuEPW9vapvKXMFrUbebqx/4I+lHOzbuv4zD0ssTE6AkcMNTDtBCFJOgJZYFMeIkZjj6rZ9N4T6Hu6 Yh84XtFl3LRGn7dTW+u+7ld7g3KB9WIRIvf5w9219+TnXzNlJlSy68AXDcPIA/KV6S0rwAf8F0tqd QqQx0HjtSnhLSIijXr1I9E8JnQwfbzY0ZKG3XSYN4T8o7BOxwYXW1pqLSwAd3SD5xqwbaxKkyoBWU 0BjPCGebzEy48hsp2XShOub3iPZPfR/xppRHVvZCwgrsGSfNe5bPYAJc+rCC1h+3mxZ5VVXoBfHzc CNL0RzMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wjXjG-0000000BD0s-0N1x; Tue, 14 Jul 2026 07:38:58 +0000 Received: from canpmsgout03.his.huawei.com ([113.46.200.218]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wjXjC-0000000BD0M-1OF6 for linux-arm-kernel@lists.infradead.org; Tue, 14 Jul 2026 07:38:55 +0000 dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=/ynr5BYYuvAAGgFG1JBPJ8ExJe4Z1HS483Kq6s4O+1k=; b=suMbfSMSr5eET3ZVh1FM9hAr5uqiOgk1m1xppsXwAY5IUm3NOxqF2FaoM91rALTLw309HWLrP AmMuYk2c9Cv7Zo+mrPwgJOjZ6tcgUaZ0a1s428qhPDNg2q/KdO645k2eT/AgyInyWEQ0wQV4RkI mfFQdB7idh+vCH5LSm1oSbA= Received: from mail.maildlp.com (unknown [172.19.162.144]) by canpmsgout03.his.huawei.com (SkyGuard) with ESMTPS id 4gzrY91gDKzpSvH; Tue, 14 Jul 2026 15:29:49 +0800 (CST) Received: from kwepemr100010.china.huawei.com (unknown [7.202.195.125]) by mail.maildlp.com (Postfix) with ESMTPS id DE52640538; Tue, 14 Jul 2026 15:38:39 +0800 (CST) Received: from [10.67.120.103] (10.67.120.103) by kwepemr100010.china.huawei.com (7.202.195.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.36; Tue, 14 Jul 2026 15:38:39 +0800 Message-ID: <9340fa94-6f26-4053-a4ca-0803af725936@huawei.com> Date: Tue, 14 Jul 2026 15:38:39 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 5/6] KVM: arm64: Add HDBSS fault handling and buffer flush To: Leonardo Bras CC: , , , , , , , , , , , , , , , , , , References: <20260709104026.2612599-1-zhengtian10@huawei.com> <20260709104026.2612599-6-zhengtian10@huawei.com> From: Tian Zheng In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.120.103] X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemr100010.china.huawei.com (7.202.195.125) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260714_003854_689563_97FD2877 X-CRM114-Status: GOOD ( 25.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 7/13/2026 10:06 PM, Leonardo Bras wrote: > On Thu, Jul 09, 2026 at 06:40:25PM +0800, Tian Zheng wrote: >> From: eillon >> >> Add HDBSS fault handling for buffer full, external abort, and general >> protection fault (GPF) events. When the HDBSS buffer becomes full, >> the hardware traps to EL2 with an HDBSSF event, which is handled by >> setting a flush request. >> >> Add kvm_flush_hdbss_buffer() to consume HDBSS buffer entries and >> propagate dirty information into the userspace-visible dirty bitmap. >> Flush is triggered on vcpu_put, check_vcpu_requests, and >> sync_dirty_log. >> >> Add esr_iss2_is_hdbssf() helper for HDBSS fault detection in guest >> abort handling. >> >> Signed-off-by: Eillon >> Signed-off-by: Tian Zheng >> --- >> arch/arm64/include/asm/esr.h | 5 +++ >> arch/arm64/include/asm/kvm_dirty_bit.h | 11 +++++ >> arch/arm64/include/asm/kvm_host.h | 1 + >> arch/arm64/kvm/arm.c | 14 ++++++ >> arch/arm64/kvm/dirty_bit.c | 62 ++++++++++++++++++++++++++ >> arch/arm64/kvm/mmu.c | 4 ++ >> 6 files changed, 97 insertions(+) >> >> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h >> index 81c17320a588..2e6b679b5908 100644 >> --- a/arch/arm64/include/asm/esr.h >> +++ b/arch/arm64/include/asm/esr.h >> @@ -437,6 +437,11 @@ >> #ifndef __ASSEMBLER__ >> #include >> >> +static inline bool esr_iss2_is_hdbssf(unsigned long esr) >> +{ >> + return ESR_ELx_ISS2(esr) & ESR_ELx_HDBSSF; > This will return a long, which will be casted as bool. > In general, what I see in the kernel is something like: > > return !!(ESR_ELx_ISS2(esr) & ESR_ELx_HDBSSF) ok! >> +} >> + >> static inline unsigned long esr_brk_comment(unsigned long esr) >> { >> return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; >> diff --git a/arch/arm64/include/asm/kvm_dirty_bit.h b/arch/arm64/include/asm/kvm_dirty_bit.h >> index 84b12f0a10af..4b28000e972f 100644 >> --- a/arch/arm64/include/asm/kvm_dirty_bit.h >> +++ b/arch/arm64/include/asm/kvm_dirty_bit.h >> @@ -10,7 +10,18 @@ >> #include >> #include >> >> +/* HDBSS entry field definitions */ >> +#define HDBSS_ENTRY_VALID BIT(0) >> +#define HDBSS_ENTRY_TTWL_SHIFT (1) >> +#define HDBSS_ENTRY_TTWL_MASK (GENMASK(3, 1)) >> +#define HDBSS_ENTRY_TTWL(x) \ >> + (((x) << HDBSS_ENTRY_TTWL_SHIFT) & HDBSS_ENTRY_TTWL_MASK) >> +#define HDBSS_ENTRY_TTWL_RESV HDBSS_ENTRY_TTWL(-4) >> +#define HDBSS_ENTRY_IPA GENMASK_ULL(55, 12) >> + >> int kvm_arm_vcpu_alloc_hdbss(struct kvm_vcpu *vcpu, unsigned int order); >> void kvm_arm_vcpu_free_hdbss(struct kvm_vcpu *vcpu); >> +void kvm_flush_hdbss_buffer(struct kvm_vcpu *vcpu); >> +int kvm_handle_hdbss_fault(struct kvm_vcpu *vcpu); >> >> #endif /* __ARM64_KVM_DIRTY_BIT_H__ */ >> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h >> index c41ec6d9c45a..cecfb884a64f 100644 >> --- a/arch/arm64/include/asm/kvm_host.h >> +++ b/arch/arm64/include/asm/kvm_host.h >> @@ -55,6 +55,7 @@ >> #define KVM_REQ_GUEST_HYP_IRQ_PENDING KVM_ARCH_REQ(9) >> #define KVM_REQ_MAP_L1_VNCR_EL2 KVM_ARCH_REQ(10) >> #define KVM_REQ_VGIC_PROCESS_UPDATE KVM_ARCH_REQ(11) >> +#define KVM_REQ_FLUSH_HDBSS KVM_ARCH_REQ(12) >> >> #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ >> KVM_DIRTY_LOG_INITIALLY_SET) >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c >> index bf6688245d83..566953a4e23a 100644 >> --- a/arch/arm64/kvm/arm.c >> +++ b/arch/arm64/kvm/arm.c >> @@ -755,6 +755,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) >> kvm_vcpu_put_hw_mmu(vcpu); >> kvm_arm_vmid_clear_active(); >> >> + if (vcpu->kvm->arch.enable_hdbss) >> + kvm_flush_hdbss_buffer(vcpu); >> + >> vcpu_clear_on_unsupported_cpu(vcpu); >> vcpu->cpu = -1; >> } >> @@ -1157,6 +1160,9 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) >> if (kvm_dirty_ring_check_request(vcpu)) >> return 0; >> >> + if (kvm_check_request(KVM_REQ_FLUSH_HDBSS, vcpu)) >> + kvm_flush_hdbss_buffer(vcpu); >> + >> check_nested_vcpu_requests(vcpu); >> } >> >> @@ -1971,7 +1977,15 @@ long kvm_arch_vcpu_unlocked_ioctl(struct file *filp, unsigned int ioctl, >> >> void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) >> { >> + /* >> + * Flush all CPUs' dirty log buffers to the dirty_bitmap. Called >> + * before reporting dirty_bitmap to userspace. Send a request with >> + * KVM_REQUEST_WAIT to flush buffer synchronously. >> + */ >> + if (!kvm->arch.enable_hdbss) >> + return; >> >> + kvm_make_all_cpus_request(kvm, KVM_REQ_FLUSH_HDBSS | KVM_REQUEST_WAIT); >> } >> >> static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm, >> diff --git a/arch/arm64/kvm/dirty_bit.c b/arch/arm64/kvm/dirty_bit.c >> index 6c7a6ef66b5a..002366337637 100644 >> --- a/arch/arm64/kvm/dirty_bit.c >> +++ b/arch/arm64/kvm/dirty_bit.c >> @@ -50,3 +50,65 @@ void kvm_arm_vcpu_free_hdbss(struct kvm_vcpu *vcpu) >> >> vcpu->arch.hdbss.hdbssbr_el2 = 0; >> } >> + >> +void kvm_flush_hdbss_buffer(struct kvm_vcpu *vcpu) >> +{ >> + int idx, curr_idx; >> + u64 *hdbss_buf; >> + struct kvm *kvm = vcpu->kvm; >> + >> + if (!kvm->arch.enable_hdbss) >> + return; >> + >> + curr_idx = HDBSSPROD_IDX(read_sysreg_s(SYS_HDBSSPROD_EL2)); >> + >> + /* Do nothing if HDBSS buffer is empty or br_el2 is NULL */ >> + if (curr_idx == 0 || vcpu->arch.hdbss.hdbssbr_el2 == 0) >> + return; >> + >> + hdbss_buf = page_address(phys_to_page(vcpu->arch.hdbss.base_phys)); >> + if (!hdbss_buf) >> + return; >> + >> + guard(write_lock_irqsave)(&vcpu->kvm->mmu_lock); >> + for (idx = 0; idx < curr_idx; idx++) { >> + u64 gpa; >> + >> + gpa = hdbss_buf[idx]; >> + if (!(gpa & HDBSS_ENTRY_VALID)) >> + continue; >> + >> + gpa &= HDBSS_ENTRY_IPA; >> + kvm_vcpu_mark_page_dirty(vcpu, gpa >> PAGE_SHIFT); > You mention that it does not support dirty-ring, but above function will > mark the page as dirty in the dirty-ring :/ > In kvm_arm_enable_hdbss_global(), we explicitly check and reject HDBSS enablement if dirty-ring is active: ``` if (kvm->dirty_ring_size)     return 0; ``` So when kvm_flush_hdbss_buffer() runs (which requires enable_hdbss = true), we know for certain that kvm->dirty_ring_size == 0. Therefore, kvm_vcpu_mark_page_dirty() will always take the dirty_bitmap path, never the dirty-ring path. That said, I'll add a comment in kvm_flush_hdbss_buffer() before dirty ring mode is supported, to make this explicit: ``` /*  * HDBSS is mutually exclusive with dirty-ring mode (see  * kvm_arm_enable_hdbss_global()), so kvm_vcpu_mark_page_dirty()  * will update the dirty_bitmap, not the dirty-ring.  */ ``` >> + } >> + >> + /* reset HDBSS index */ >> + write_sysreg_s(0, SYS_HDBSSPROD_EL2); >> + vcpu->arch.hdbss.hdbssprod_el2 = 0; >> + isb(); >> +} >> + >> +int kvm_handle_hdbss_fault(struct kvm_vcpu *vcpu) >> +{ >> + u64 prod; >> + u64 fsc; >> + >> + prod = read_sysreg_s(SYS_HDBSSPROD_EL2); >> + fsc = FIELD_GET(HDBSSPROD_EL2_FSC_MASK, prod); >> + >> + switch (fsc) { >> + case HDBSSPROD_EL2_FSC_OK: >> + /* Buffer full, set request to flush on next vcpu exit */ >> + kvm_make_request(KVM_REQ_FLUSH_HDBSS, vcpu); >> + return 1; >> + case HDBSSPROD_EL2_FSC_ExternalAbort: >> + case HDBSSPROD_EL2_FSC_GPF: >> + return -EFAULT; >> + default: >> + /* Unknown fault. */ >> + WARN_ONCE(1, >> + "Unexpected HDBSS fault type, FSC: 0x%llx (prod=0x%llx, vcpu=%d)\n", >> + fsc, prod, vcpu->vcpu_id); >> + return -EFAULT; >> + } >> +} >> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c >> index 83251d95bf3f..949fb895add6 100644 >> --- a/arch/arm64/kvm/mmu.c >> +++ b/arch/arm64/kvm/mmu.c >> @@ -2242,6 +2242,7 @@ int kvm_handle_guest_sea(struct kvm_vcpu *vcpu) >> } >> >> /** >> + >> * kvm_handle_guest_abort - handles all 2nd stage aborts >> * @vcpu: the VCPU pointer >> * >> @@ -2279,6 +2280,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) >> >> is_iabt = kvm_vcpu_trap_is_iabt(vcpu); >> >> + if (esr_iss2_is_hdbssf(esr)) >> + return kvm_handle_hdbss_fault(vcpu); >> + >> if (esr_fsc_is_translation_fault(esr)) { >> /* Beyond sanitised PARange (which is the IPA limit) */ >> if (fault_ipa >= BIT_ULL(get_kvm_ipa_limit())) { >> -- >> 2.33.0 >>