From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B975C636D4 for ; Fri, 10 Feb 2023 19:22:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AImddxPDjsqmcTUF2b1iFX6LXSkjzorT2bA9Uhc6/Ik=; b=DjwzHis2zXU+k2 MfuZrPFvPxrQ6ivIsBXXYUS5eyL1B81MZXj9HjJe2MX13hQX3vTn4a+uDxVlNWm1jqgjrNd7n3j39 cmUT8AGZoJXrrtIySNt+QsheAW+/KFT1y4xVDAneSIWISHpfk4oABJYEVWLC9FtxxXO3QfltcSQGS szRMW4sOw8MSfkGU6yS+2oOMOIpFDyj8U43wn1tlzxHN45oiy+kolIaEwflS+omUkKJxnJyfwS0fn oed5wnIrOqH0Qhbqte6LaDw8TGrswYAaUw9grqLHovaCd9rUu1L9HRlefAieCHv3KNL2glYcJGfHN 70z1NZYTJQB8KyZ2io6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pQYxg-0074gU-63; Fri, 10 Feb 2023 19:21:32 +0000 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pQYxc-0074eW-MI for linux-arm-kernel@lists.infradead.org; Fri, 10 Feb 2023 19:21:30 +0000 Received: by mail-wr1-x431.google.com with SMTP id r2so6025535wrv.7 for ; Fri, 10 Feb 2023 11:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1Tz1WhSiDyiHb7swgsfraxGg6wWStHRX2Amb3b1VcmU=; b=WKCPMt8G4P6kglvBEYuR6Y8VGe4vLKREbo46B1FzP36qdC9o/gXh5YU6gEW7L3708X lrxXAzEPB1j3pGDdzQh1aduk3Z9ZrGueXQk2asJVFXfbphcdAajT2cuL8SDpesGlCuT5 zP27G5KvohxP/B+J2p8XFakN/0ZUkhWnnBrHWiIoZRCeISeKJDx0vx5qCieGs4EnxLMv DR+bECwo2Frc9+Tg0Sq3Y8jMTdALTUIhprnoG8OmKK9vaJW9Bl1ofg1Vnea3kae6KK3A RYJiA29h4m4QGtT3sklUElmYPdchtrnZQvS2mzkuO+xv8NS8SwBva8yxv2T57qImR1fr WyNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=1Tz1WhSiDyiHb7swgsfraxGg6wWStHRX2Amb3b1VcmU=; b=CFrHZD1LifHsmLUGFeAyWpY2zBTDJU9pznQQOflAPwrnqXrZ7040N5uUZ51aj3MDzb ehD7eJm1UmeqjO6NRGVJWNQBvGvbgtBSdNFl2O2Kr2Xye6dlxMomVuCv+7ZOQRND2BrR GCSi9+6l7GmEUR1h6Ueynu4O9SeRyZcwr0fWLapchXPxzmjIJSFyD/VBeBtOJl81lMwV hG+XbJ0LgKlDK1Z2xo5N9hsSEDEXDoCyw63Py4Or9/LMyE/JVuyR2F4TJ9b4mPiAMoCz LvJkFpKd+Auo3udRB90chBeOv4ZOV/l/jBV7oGPMiNFCTfrrV0dGAFIYm9aXYdbcCOmd xPuQ== X-Gm-Message-State: AO0yUKXMiu/vafZZs9ZaRtSNSNQi0DbmbLJ8oCPghlaBLVdH0/PprFCm SfP42aE2b+ikkosktkwZPYTdPg== X-Google-Smtp-Source: AK7set81Xt2KccZQRdMWDTRiY2Gd0tbVvECaL276b9b5J5FRXHU0WwGuCEaGmrLLiiQcY/fFP5oI7g== X-Received: by 2002:adf:f212:0:b0:2c3:dbe0:58ea with SMTP id p18-20020adff212000000b002c3dbe058eamr16025908wro.47.1676056883588; Fri, 10 Feb 2023 11:21:23 -0800 (PST) Received: from myrica (054592b0.skybroadband.com. [5.69.146.176]) by smtp.gmail.com with ESMTPSA id y1-20020a5d6201000000b002c3ea5ebc73sm4253499wru.101.2023.02.10.11.21.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 11:21:23 -0800 (PST) Date: Fri, 10 Feb 2023 19:21:14 +0000 From: Jean-Philippe Brucker To: Mostafa Saleh Cc: maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, joro@8bytes.org, robin.murphy@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, dbrazdil@google.com, ryan.roberts@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Subject: Re: [RFC PATCH 15/45] KVM: arm64: pkvm: Add __pkvm_host_share/unshare_dma() Message-ID: References: <20230201125328.2186498-1-jean-philippe@linaro.org> <20230201125328.2186498-16-jean-philippe@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230210_112128_776610_1278C67D X-CRM114-Status: GOOD ( 29.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Feb 07, 2023 at 12:53:56PM +0000, Mostafa Saleh wrote: > > +static int __host_check_page_dma_shared(phys_addr_t phys_addr) > > +{ > > + int ret; > > + u64 hyp_addr; > > + > > + /* > > + * The page is already refcounted. Make sure it's owned by the host, and > > + * not part of the hyp pool. > > + */ > > + ret = __host_check_page_state_range(phys_addr, PAGE_SIZE, > > + PKVM_PAGE_SHARED_OWNED); > > + if (ret) > > + return ret; > > + > > + /* > > + * Refcounted and owned by host, means it's either mapped in the > > + * SMMU, or it's some VM/VCPU state shared with the hypervisor. > > + * The host has no reason to use a page for both. > > + */ > > + hyp_addr = (u64)hyp_phys_to_virt(phys_addr); > > + return __hyp_check_page_state_range(hyp_addr, PAGE_SIZE, PKVM_NOPAGE); > > This works for hyp-host sharing, I am worried about scalability of this. > For example FFA(still on the list) adds a new entity that can share data > with host, and we would need an extra check for it. > https://lore.kernel.org/kvmarm/20221116170335.2341003-1-qperret@google.com/ Right, although it looks like the ff-a support doesn't need a refcount at the moment so passing such a page to iommu_map() would fail at check_share() (but I may be wrong, I'll need another look). Even if it did use a refcount, I think it may be fine to share it between different entities: the refcount ensures that every sharing is undone before the page can be donated to another entity, and a separate tracking ensures that the host undoes the right thing (the io-pgtable mappings for pages shared with the IOMMU, and the secure world for ff-a) Regardless I agree that this doesn't scale, it gets too complex and I'd prefer if tracking the page state was less subtle. I'll try to find something more generic. > > One way I can think about this, is to use the SW bits in SMMU page table to > represent ownership, for example for PKVM_ID_IOMMU > do_share: > -completer: set SW bits in the SMMU page table to > PKVM_PAGE_SHARED_BORROWED I'm not sure I understand, would we add this in order to share a page that is already mapped in the SMMU with another entity (e.g. ff-a)? Doing it this way would be difficult because there may be a lot of different SMMU page tables mapping the page. The fact that a page is mapped in the SMMU page table already indicates that it's borrowed from the host so the SW bits seem redundant. > > do_unshare: > -completer: set SW bit back to PKVM_PAGE_OWNED(I think this is good > enough for host ownership) > > In __pkvm_host_share_dma_page > If page refcount > 1, it succeeds if only the page was borrowed in the > SMMU page table and shared in the host page table. We could also move more information into the vmemmap, because struct hyp_page still has some space for page state. It has six free bits at the moment, which is enough for both owner ID and share state, and maybe we could steal a couple more from the refcount and order fields if necessary. With that the iommu_map() and unmap() path could also be made a lot more scalable because they wouldn't need to take any page table locks when updating page state, a cmpxchg may be sufficient. Thanks, Jean _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel