From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BBA1C4361B for ; Fri, 18 Dec 2020 08:24:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2CE4823A03 for ; Fri, 18 Dec 2020 08:24:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2CE4823A03 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=APIGmaXPPo8EV29HiWAL/0azZBhEEVt1bLjGCPMs4CQ=; b=HRzGVbu/EYfotHpiTgPUbb4hz KtWOCADkxQIuPWSqglUiV0YEKprJDVOXmjgXwLg34fulWpV202I82Tw9Q53KWk7X4mcPdomXEJ+x8 HPL9Zk15CVXOU1BOYufCE9vgpkSXkHGoADEjEwKNUF9FRKmDakBHWw5kgb8wQgVHSmumBp1L5Ruz5 +m2vVTJt15Ow4PzwnHlyLh0lkk15NWItZAvjMqHVr+aUsn3hD4A938S+DIjvzsBbGi4orjS/zU9nB bkruGBQ+WCKSFa9BIhjNJSAi/8FASNOtPP9kQWtZpSXML5l8c/DQo1F3ijeOKg0Hy+qUKmi5Lb3u6 rDDytX7YQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kqB2N-0004cW-A7; Fri, 18 Dec 2020 08:22:55 +0000 Received: from szxga07-in.huawei.com ([45.249.212.35]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kqB1w-0004YS-V9 for linux-arm-kernel@lists.infradead.org; Fri, 18 Dec 2020 08:22:53 +0000 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4Cy1zZ3NMpz7Gx1; Fri, 18 Dec 2020 16:21:30 +0800 (CST) Received: from [10.174.187.37] (10.174.187.37) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.498.0; Fri, 18 Dec 2020 16:21:59 +0800 Subject: Re: [PATCH 4/7] vfio: iommu_type1: Fix missing dirty page when promote pinned_scope To: Alex Williamson References: <20201210073425.25960-1-zhukeqian1@huawei.com> <20201210073425.25960-5-zhukeqian1@huawei.com> <20201214170459.50cb8729@omen.home> <20201215085359.053e73ed@x1.home> From: Keqian Zhu Message-ID: <340a58c3-3781-db31-59fa-06b015d27a5e@huawei.com> Date: Fri, 18 Dec 2020 16:21:58 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <20201215085359.053e73ed@x1.home> X-Originating-IP: [10.174.187.37] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201218_032252_244292_8AE1F3F0 X-CRM114-Status: GOOD ( 14.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jiangkunkun@huawei.com, Andrew Morton , kvm@vger.kernel.org, Suzuki K Poulose , Marc Zyngier , Joerg Roedel , Cornelia Huck , linux-kernel@vger.kernel.org, Sean Christopherson , Alexios Zavras , iommu@lists.linux-foundation.org, Mark Brown , James Morse , Julien Thierry , Catalin Marinas , wanghaibin.wang@huawei.com, Thomas Gleixner , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, Robin Murphy Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2020/12/15 23:53, Alex Williamson wrote: > On Tue, 15 Dec 2020 17:37:11 +0800 > zhukeqian wrote: > >> Hi Alex, >> >> On 2020/12/15 8:04, Alex Williamson wrote: [...] >>>> >>>> +static void vfio_populate_bitmap_all(struct vfio_iommu *iommu) >>>> +{ >>>> + struct rb_node *n; >>>> + unsigned long pgshift = __ffs(iommu->pgsize_bitmap); >>>> + >>>> + for (n = rb_first(&iommu->dma_list); n; n = rb_next(n)) { >>>> + struct vfio_dma *dma = rb_entry(n, struct vfio_dma, node); >>>> + unsigned long nbits = dma->size >> pgshift; >>>> + >>>> + if (dma->iommu_mapped) >>>> + bitmap_set(dma->bitmap, 0, nbits); >>>> + } >>>> +} >>> >>> >>> If we detach a group which results in only non-IOMMU backed mdevs, >>> don't we also clear dma->iommu_mapped as part of vfio_unmap_unpin() >>> such that this test is invalid? Thanks, >> >> Good spot :-). The code will skip bitmap_set under this situation. >> >> We should set the bitmap unconditionally when vfio_iommu is promoted, >> as we must have IOMMU backed domain before promoting the vfio_iommu. >> >> Besides, I think we should also mark dirty in vfio_remove_dma if dirty >> tracking is active. Right? > > There's no remaining bitmap to mark dirty if the vfio_dma is removed. > In this case it's the user's responsibility to collect remaining dirty > pages using the VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP support in the > VFIO_IOMMU_UNMAP_DMA ioctl. Thanks, > Hi Alex, Thanks for pointing it out. I also notice that vfio_iommu_type1_detach_group will remove all dma_range (in vfio_iommu_unmap_unpin_all). If this happens during dirty tracking, then we have no chance to report dirty log to userspace. Besides, we will add more dirty log tracking ways to VFIO definitely, but we has no framework to support this, thus makes it inconvenient to extend and easy to lost dirty log. Giving above, I plan to refactor our dirty tracking code. One core idea is that we should distinguish Dirty Range Limit (such as pin, fully dirty) and Real Dirty Track (such as iopf, smmu httu). Thanks, Keqian _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel