From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A81CC2BD09 for ; Mon, 24 Jun 2024 16:53:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9iNykJAzove/zf+ZXfa+Ojht8siTvjot0NY03mffCkE=; b=Wj+0kEaiRNONNZ/wJETfyNW9R4 88Dvz3vyxmb63OiWlo+NXs6aYpyrxQ3g2aJXKLyHKNHYeOvFXbOH78imWZCdv0lfmiZ1jkTleRJMK 8zmzsCOYo0qbsTJyG9SjGBXNszbcJbQgH+IsjvFArnAGHz9Cd1Td5LG57FEVYjDr0vdhTW6GbZCWt zjPASnVVbtour23E9/2VA6jNPRy3Z1kXcMom4RgNUwDaFbbj9WI+JLI/ym6moQQQssVR9Z7tNxrxJ RynhBuUWDtrXc0l0+hFpSOjErW2uCiuuZRiRfTgwiu2cMZV4WKoatdmLGh927DjYqagXe9S108gGN jbjhO2Sw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLmwI-000000000AW-1b2H; Mon, 24 Jun 2024 16:53:10 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLmwB-0000000009i-3WY9 for linux-arm-kernel@lists.infradead.org; Mon, 24 Jun 2024 16:53:05 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e02a4de4f4eso9229707276.1 for ; Mon, 24 Jun 2024 09:53:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1719247982; x=1719852782; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9iNykJAzove/zf+ZXfa+Ojht8siTvjot0NY03mffCkE=; b=OpCWZWzNT/tPXOWiqmeryTr1GOs+t9Y3wphEglMnOneAWi5zyjqgaJA0tsch+5ISoX cqkvOoTm4MU8is8wdKf6uOfX7rPn3sNGZtyvaFOcYJy8wnEzQWmc8Tj0MMbcPMPwnnRB 3hbXkFYyQmB2CBEJ0lqaNA/2iae1WXIxuhKDenrQ46MdOdkBAUW4NQNJQt6vR+U9275h 0JdRl9LkFBF1+2fOA0yJOEJn+POoVLMXQtR9DeKv5hiO40Lue8vdNRjVl08y54ZcKPKP aRAFBx6cl8rw7uUe6+XFc49Zp1ygqS366/DN8EZsxCMa0VGunzTBrCyb4M+c+ov1QeLt QqFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719247982; x=1719852782; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9iNykJAzove/zf+ZXfa+Ojht8siTvjot0NY03mffCkE=; b=UYjpKMpNQCekQQrPVwi0bQJIH6O2MMbqY0m2RQt6S7tz1yTswh//cvkDLjFIz2KzXX 0BT+lHpXVDusDM4K6qQIbJL4G+v+0piVggAzQYH6/VHnFakUcZTM184unOWH6Uk7Fc/u nQlMNLnTY1JtOheireW11tEhRZT0u3oaqjk81lnMtI8uxnSOKZ1fejKJFTM0Nym+qZV4 bfrytMIJOnLOVQGAToJ+2mWXDwy23Jx+D+kKb3eTg8SSNfagGGbTVb7kh7UeoEVnqoXN De4YBVO7DIKE7XZD8z+ndlxZ7cqJyLS1lIFQ4wgl8+ZEoTfV5GC2lnPKQduyC1ocRPob cBTQ== X-Forwarded-Encrypted: i=1; AJvYcCWF7IGzTfYB2EeOFOqXW0KAuZ8+yGe0zvfjainAFwPvoGp83UI5Htd3s1J5RA1p2aoylyDLOOd7PkOwBpGXIy0yAAvyOIYz+SJOqAwKHT8WZwMAO0g= X-Gm-Message-State: AOJu0YyIoYY9l1VRN5wJsyRyy5QJIFSpy4Xnn6s4n/ZVV0ndltzIlrPh i2stBAxTAQbz+rFsvyaV2Kuu6myc0wh17IDO9JoGmHvKvMHzePurLYiTfsmUU18Tvi+o+CqgZWl 86A== X-Google-Smtp-Source: AGHT+IEPXu/gKdcESrzvh7SoSsa9p93Pm1qfCCz4PS8Ka0QVRbNxNeO8As2woYzjP4FbCutHvtfSiZgzhIQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:2b04:b0:df2:eaac:d811 with SMTP id 3f1490d57ef6-e02fc598947mr18902276.10.1719247981721; Mon, 24 Jun 2024 09:53:01 -0700 (PDT) Date: Mon, 24 Jun 2024 09:53:00 -0700 In-Reply-To: <20240208154210.GP31743@ziepe.ca> Mime-Version: 1.0 References: <20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com> <20240208151837.35068-5-shameerali.kolothum.thodi@huawei.com> <20240208154210.GP31743@ziepe.ca> Message-ID: Subject: Re: [RFC PATCH v2 4/7] iommufd: Associate kvm pointer to iommufd ctx From: Sean Christopherson To: Jason Gunthorpe Cc: Shameer Kolothum , kvmarm@lists.linux.dev, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxarm@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com, maz@kernel.org, oliver.upton@linux.dev, will@kernel.org, robin.murphy@arm.com, jean-philippe@linaro.org, jonathan.cameron@huawei.com Content-Type: text/plain; charset="us-ascii" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_095303_906001_795C21A5 X-CRM114-Status: GOOD ( 25.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Feb 08, 2024, Jason Gunthorpe wrote: > On Thu, Feb 08, 2024 at 03:18:34PM +0000, Shameer Kolothum wrote: > > > diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h > > index 991f864d1f9b..28ede82bb1a6 100644 > > --- a/drivers/iommu/iommufd/iommufd_private.h > > +++ b/drivers/iommu/iommufd/iommufd_private.h > > @@ -16,6 +16,7 @@ struct iommu_domain; > > struct iommu_group; > > struct iommu_option; > > struct iommufd_device; > > +struct kvm; > > > > struct iommufd_ctx { > > struct file *file; > > @@ -27,6 +28,8 @@ struct iommufd_ctx { > > /* Compatibility with VFIO no iommu */ > > u8 no_iommu_mode; > > struct iommufd_ioas *vfio_ioas; > > + /* Associated KVM pointer */ > > + struct kvm *kvm; > > }; > > Associating the KVM with the entire iommufd is a big hammer, is this > what we want to do? > > I know it has to be linked to domain allocation and the coming > "viommu" object, and it is already linked to VFIO. > > It means we support one KVM per iommufd (which doesn't seem > unreasonable, but also the first time we've had such a limitation) And if KVM+iommufd come as pairs, wouldn't iommufd_ctx_set_kvm() need to reject attempts to bind devices associated with different KVMs, as opposed to silently ignoring that case? E.g. something like the below? Or is there magic elsewhere in the stack that prevents such a scenario? void iommufd_ctx_set_kvm(struct iommufd_ctx *ictx, struct kvm *kvm) { int r = 0; xa_lock(&ictx->objects); if (!ictx->kvm) ictx->kvm = kvm; else if (icxt->kvm != kvm) r = -EINVAL; xa_unlock(&ictx->objects); } > The other option would be to pass in the kvm to the individual sub > objects. > > Kevin? > > Sean would you be OK with this approach considering your other series > to try to make more of this private? Sorry, I completely missed this. If kvm_pinned_vmid_{get,put}() are implemented directly by KVM ARM, then I don't have any immediate concerns, as KVM ARM is a long, long way from being able to isolate KVM from the core kernel. And if we ever want/need to provide generic APIs for propagating state from KVM into iommufd, bundling the KVM file pointer[*] with a set of function pointers wouldn't be terribly difficult. That said, I find the on-demand pinning to be very odd. IIUC, if KVM runs out of pinnable VMIDs, attaching a device to the KVM+iommu will fail. Failing an iommufd operation because of a (potentially transient) KVM resource issue is rather unpleasant. And assuming that pinnable VMIDs are a somewhat scarce resource, it wouldn't suprise me if someone wanted to add cgroup integration, e.g. similar to the misc cgroup that's used to manage SEV(-ES) ASIDs on KVM AMD (IIUC, an SEV ASID is analagous to an ARM VMID). Rather than on-demand pinning, would it make sense to have KVM provide an ioctl() (or capability, or VM type) to let userspace pin a VM's VMID? That would allow for a much saner failure mode, and I suspect would be cleaner in general for iommufd. [*] https://lore.kernel.org/all/ZXkVSKULLivrMkBl@google.com