From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 155BCD232EA for ; Fri, 9 Jan 2026 07:53:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:CC:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cAR5LxUosn5MR+Z/5Moz9cTm0bb98yOxJOET8rzyKpQ=; b=U34GFW9YX5XgHL/3c5Wk6waZei XVj3q1jfscPpYdUtz0KkG8FqJGoA5UxdqvTSrG3h8jHSGjknYs2Kttk9kKyTWGpepczxSO7XIdqlQ JlkPCx/kMWuX2u7f1PJ0FwyS4utGT+mOI+O1xWvpAqgDs/o+ysTY1pVK8iE5ZSNzLnuKO0gbh0/Bp oRRp3CG6NxfCzS5F52sEOEKfAgD+9p3rFliHaPFrTF5FDMBh/+dB+H8gpT51+K92GHPrbHEzNAC12 fVVA/fDAiw1FoElTZ9oLaKsYJnEN8zlKo8ksLnud0cv2pMRmhSbPekvOAXrvKdycFysfaIBGavgk0 fDyfqQgg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1ve7It-00000001nFs-3N5r; Fri, 09 Jan 2026 07:53:03 +0000 Received: from canpmsgout01.his.huawei.com ([113.46.200.216]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1ve7Iq-00000001nE8-40Yh for linux-arm-kernel@lists.infradead.org; Fri, 09 Jan 2026 07:53:02 +0000 dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=cAR5LxUosn5MR+Z/5Moz9cTm0bb98yOxJOET8rzyKpQ=; b=dLEEdksaplpS8/45BsNH/BlSmJya5HWO6a3QfmnzOgvITtGrcS4tQqOlz5Stt/MisPFZViHE/ WVF9s/WF84xFLu/zuaPkqf+n3pKgW1aAEhkIbKtAQoi0MAiecbB4nh+i3PEjuBou0htiMVD6AY+ 8YHqy1c3T8XqlD1qRKeDwVE= Received: from mail.maildlp.com (unknown [172.19.163.0]) by canpmsgout01.his.huawei.com (SkyGuard) with ESMTPS id 4dnYn92STTz1T4GT; Fri, 9 Jan 2026 15:49:01 +0800 (CST) Received: from kwepemr100010.china.huawei.com (unknown [7.202.195.125]) by mail.maildlp.com (Postfix) with ESMTPS id 13CD74036C; Fri, 9 Jan 2026 15:52:47 +0800 (CST) Received: from [10.67.120.103] (10.67.120.103) by kwepemr100010.china.huawei.com (7.202.195.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.36; Fri, 9 Jan 2026 15:52:46 +0800 Message-ID: <36e0a4d0-b440-4aba-8dfd-0c0fcb5f4318@huawei.com> Date: Fri, 9 Jan 2026 15:52:45 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 4/5] KVM: arm64: Enable HDBSS support and handle HDBSSF events To: Robert Hoo , , , , , , CC: , , , , , , , , , , , References: <20251121092342.3393318-1-zhengtian10@huawei.com> <20251121092342.3393318-5-zhengtian10@huawei.com> <87df4cba-b191-49cf-9486-fc379470a6eb@gmail.com> <77111894-1b9f-4970-b41f-48e3a4c4b754@gmail.com> From: Tian Zheng In-Reply-To: <77111894-1b9f-4970-b41f-48e3a4c4b754@gmail.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.120.103] X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemr100010.china.huawei.com (7.202.195.125) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260108_235301_607745_80663706 X-CRM114-Status: GOOD ( 20.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 12/28/2025 9:21 PM, Robert Hoo wrote: > On 12/24/2025 2:15 PM, Tian Zheng wrote: >> >> >> On 12/17/2025 9:39 PM, Robert Hoo wrote: >>> On 11/21/2025 5:23 PM, Tian Zheng wrote: >>>> From: eillon >>>> >>>> Implement the HDBSS enable/disable functionality using the >>>> KVM_CAP_ARM_HW_DIRTY_STATE_TRACK ioctl. >>>> >>>> Userspace (e.g., QEMU) can enable HDBSS by invoking the ioctl >>>> at the start of live migration, configuring the buffer size. >>>> The feature is disabled by invoking the ioctl again with size >>>> set to 0 once migration completes. >>>> >>>> Add support for updating the dirty bitmap based on the HDBSS >>>> buffer. Similar to the x86 PML implementation, KVM flushes the >>>> buffer on all VM-Exits, so running vCPUs only need to be kicked >>>> to force a VM-Exit. >>>> >>>> Signed-off-by: eillon >>>> Signed-off-by: Tian Zheng >>>> --- >>>>   arch/arm64/include/asm/kvm_host.h |  10 +++ >>>>   arch/arm64/include/asm/kvm_mmu.h  |  17 +++++ >>>>   arch/arm64/kvm/arm.c              | 107 >>>> ++++++++++++++++++++++++++++++ >>>>   arch/arm64/kvm/handle_exit.c      |  45 +++++++++++++ >>>>   arch/arm64/kvm/hyp/vhe/switch.c   |   1 + >>>>   arch/arm64/kvm/mmu.c              |  10 +++ >>>>   arch/arm64/kvm/reset.c            |   3 + >>>>   include/linux/kvm_host.h          |   1 + >>>>   8 files changed, 194 insertions(+) >>>> >>>> diff --git a/arch/arm64/include/asm/kvm_host.h >>>> b/arch/arm64/include/ asm/kvm_host.h >>>> index d962932f0e5f..408e4c2b3d1a 100644 >>>> --- a/arch/arm64/include/asm/kvm_host.h >>>> +++ b/arch/arm64/include/asm/kvm_host.h >>>> @@ -87,6 +87,7 @@ int __init kvm_arm_init_sve(void); >>>>   u32 __attribute_const__ kvm_target_cpu(void); >>>>   void kvm_reset_vcpu(struct kvm_vcpu *vcpu); >>>>   void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu); >>>> +void kvm_arm_vcpu_free_hdbss(struct kvm_vcpu *vcpu); >>>> >>>>   struct kvm_hyp_memcache { >>>>       phys_addr_t head; >>>> @@ -793,6 +794,12 @@ struct vcpu_reset_state { >>>>       bool        reset; >>>>   }; >>>> >>>> +struct vcpu_hdbss_state { >>>> +    phys_addr_t base_phys; >>>> +    u32 size; >>>> +    u32 next_index; >>>> +}; >>>> + >>>>   struct vncr_tlb; >>>> >>>>   struct kvm_vcpu_arch { >>>> @@ -897,6 +904,9 @@ struct kvm_vcpu_arch { >>>> >>>>       /* Per-vcpu TLB for VNCR_EL2 -- NULL when !NV */ >>>>       struct vncr_tlb    *vncr_tlb; >>>> + >>>> +    /* HDBSS registers info */ >>>> +    struct vcpu_hdbss_state hdbss; >>>>   }; >>>> >>>>   /* >>>> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/ >>>> asm/kvm_mmu.h >>>> index e4069f2ce642..6ace1080aed5 100644 >>>> --- a/arch/arm64/include/asm/kvm_mmu.h >>>> +++ b/arch/arm64/include/asm/kvm_mmu.h >>>> @@ -331,6 +331,23 @@ static __always_inline void >>>> __load_stage2(struct kvm_s2_mmu *mmu, >>>>       asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); >>>>   } >>>> >>>> +static __always_inline void __load_hdbss(struct kvm_vcpu *vcpu) >>>> +{ >>>> +    struct kvm *kvm = vcpu->kvm; >>>> +    u64 br_el2, prod_el2; >>>> + >>>> +    if (!kvm->enable_hdbss) >>>> +        return; >>>> + >>>> +    br_el2 = HDBSSBR_EL2(vcpu->arch.hdbss.base_phys, vcpu- >>>> >arch.hdbss.size); >>>> +    prod_el2 = vcpu->arch.hdbss.next_index; >>>> + >>>> +    write_sysreg_s(br_el2, SYS_HDBSSBR_EL2); >>>> +    write_sysreg_s(prod_el2, SYS_HDBSSPROD_EL2); >>>> + >>>> +    isb(); >>>> +} >>>> + >>>>   static inline struct kvm *kvm_s2_mmu_to_kvm(struct kvm_s2_mmu *mmu) >>>>   { >>>>       return container_of(mmu->arch, struct kvm, arch); >>>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c >>>> index 870953b4a8a7..64f65e3c2a89 100644 >>>> --- a/arch/arm64/kvm/arm.c >>>> +++ b/arch/arm64/kvm/arm.c >>>> @@ -79,6 +79,92 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu >>>> *vcpu) >>>>       return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; >>>>   } >>>> >>>> +void kvm_arm_vcpu_free_hdbss(struct kvm_vcpu *vcpu) >>>> +{ >>>> +    struct page *hdbss_pg = NULL; >>>> + >>>> +    hdbss_pg = phys_to_page(vcpu->arch.hdbss.base_phys); >>>> +    if (hdbss_pg) >>>> +        __free_pages(hdbss_pg, vcpu->arch.hdbss.size); >>>> + >>>> +    vcpu->arch.hdbss = (struct vcpu_hdbss_state) { >>>> +        .base_phys = 0, >>>> +        .size = 0, >>>> +        .next_index = 0, >>>> +    }; >>>> +} >>>> + >>>> +static int kvm_cap_arm_enable_hdbss(struct kvm *kvm, >>>> +                    struct kvm_enable_cap *cap) >>>> +{ >>>> +    unsigned long i; >>>> +    struct kvm_vcpu *vcpu; >>>> +    struct page *hdbss_pg = NULL; >>>> +    int size = cap->args[0]; >>>> +    int ret = 0; >>>> + >>>> +    if (!system_supports_hdbss()) { >>>> +        kvm_err("This system does not support HDBSS!\n"); >>>> +        return -EINVAL; >>>> +    } >>>> + >>>> +    if (size < 0 || size > HDBSS_MAX_SIZE) { >>>> +        kvm_err("Invalid HDBSS buffer size: %d!\n", size); >>>> +        return -EINVAL; >>>> +    } >>>> + >>> >>> I think you should check if it's already enabled here. What if user >>> space calls this twice? >> >> Ok, I review the implement of qemu, when disable the hdbss feature in >> ram_save_cleanup, size=0 will be set, so here can add a check, if (size >> && kvm->arch.enable_hdbss), we will do nothing. >> > > I mean you should check if ' kvm->enable_hdbss' is already set, if so, > return rather than alloc_pages() in below (you have allocated in > previous call with valid 'size'). > > qemu is just one of the user space applications that would possibly > call this API, you cannot rely on your qemu patch's flow/sequence as > assumption to design a KVM API's implementation. Yes, The latest v3 patch fixes this bug by checking if (size > 0 && kvm->arch.enable_hdbss). When this condition is met, the function returns immediately rather than alloc_pages(). > >>> >>>> +    /* Enable the HDBSS feature if size > 0, otherwise disable it. */ >>>> +    if (size) { >>>> +        kvm_for_each_vcpu(i, vcpu, kvm) { >>>> +            hdbss_pg = alloc_pages(GFP_KERNEL_ACCOUNT, size); >>>> +            if (!hdbss_pg) { >>>> +                kvm_err("Alloc HDBSS buffer failed!\n"); >>>> +                ret = -ENOMEM; >>>> +                goto error_alloc; >>>> +            } >>>> + >>>> +            vcpu->arch.hdbss = (struct vcpu_hdbss_state) { >>>> +                .base_phys = page_to_phys(hdbss_pg), >>>> +                .size = size, >>>> +                .next_index = 0, >>>> +            }; >>>> +        } >>>> + >>>> +        kvm->enable_hdbss = true; >>>> +        kvm->arch.mmu.vtcr |= VTCR_EL2_HD | VTCR_EL2_HDBSS; >>> > >