From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70A1BCD13CF for ; Tue, 3 Sep 2024 09:02:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mcAweApl0SMnSEwpMO3VeLlceofmGojlXD55ghcyISI=; b=cO1xBxJ/GDGlb0RI3+JRDRLlx+ Owtph9EpRTYsvmAKEwkrBJzWKpg7JzDEi/eIPhTd0fQHOtx+ES+9KwjBpTwxGwF+1y6jyjw7DXGMy XtSO8FQPjbfmb2QWNZJpJBM43ihlnqvms/OkTOqUSllmhCr3sXu5zgcKcpFDsyzoEH+ttmn0UnGMv 5EfPhPOtYkQ1d0rmTSrkF6O7UaK3KiqKVClY1pwjszmHZPVLyvcFmCIhv8IOrRTTkgm79EanoQrX7 zT58MVtWBHsuVoUumeSUyjFT0K61mt8+cnwzaTmnKHPKFA0EBl+N9FVWWVM4sACwBIMD7jTtKq1BF 7NjDVH7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1slPQ4-0000000H2PD-22aF; Tue, 03 Sep 2024 09:01:48 +0000 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1slPOy-0000000H24F-2TpZ for linux-arm-kernel@lists.infradead.org; Tue, 03 Sep 2024 09:00:42 +0000 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-42bba6a003bso109165e9.0 for ; Tue, 03 Sep 2024 02:00:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725354038; x=1725958838; darn=lists.infradead.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=mcAweApl0SMnSEwpMO3VeLlceofmGojlXD55ghcyISI=; b=aWEsQWCtJP1Jd6a1NOKeanv3o70cz8Znwx2WTd2nsOi3DntoHOWiMaY3syNF5JgTZQ 2zyYIUUMvRSAzVb6NAChgA4dMfM/1LF7mEdwZS+TKqeuFPLX0R4XAZdAIjNzZ/qjSsVh NsI3HS849lg5h/i7kSXu8X6qEGMN1m4Ut9+6TpRvckghL3KPJettM/1KraVPLneGi1Kd rh/UWZjCPlL2VKanvz/3/6qU0IU7TUeG8pbI8NtID0+jiODN4xDaiGUqxxA6pKBJsKyC SmsJSTPtg3tF9vLRqugtGOBOwmnAb+GXOySq79Ohharnc/2ofFQTMLc+CVeF72DRaxSF RsFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725354038; x=1725958838; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mcAweApl0SMnSEwpMO3VeLlceofmGojlXD55ghcyISI=; b=JhD5GHpvsvOpyvU9juNRLiWgLB2DXYfhy3RLY81afcm0n6i5iNZ5xZETLi5gt/CPTG 33Dv8iVmGDLL8feTTeo5pwW9HaM1yJLa9WNoAPS2ImKlws5H+cgrOF9QNQc+Ez1nGfnQ Eia3SxQSjqaLKFobE5pQhYLV/XhJRR0BurQXrQ2qZDb/uuiCrNx1epiAvru7Pd0AdZ0L 0MfySNl7f8iZFb2T9gdMbM8S/FDi/8/3vDlWYZVXpdSDuHGIc4V617AqY2ixjJkOpVRb cGpeah0pB2D/QnaG0VMo5f2H+WiFRy24pyBz5LqNn4NV1mjnTLVuBnpepUuF1wniPWZ0 0bZA== X-Forwarded-Encrypted: i=1; AJvYcCXdiARsGANznBOTvV/pdnZ948VhADyu55Zx7RQS4KFZ0s9eET7Ozje0ZQfCUt1k++iAieUSb6td8vyCbA357Vu4@lists.infradead.org X-Gm-Message-State: AOJu0Yz9Fiut3JU99hrn13Fn6we1JtASQdRlfiRA2Q/YZ0678asBcakI TXaHlcTOtssmkCENnpoi/WTScwY4o4PaQEtn1TfXQPi7riaZh29vi1a7ut4kx4pvxapd5bR6xo5 s1Q== X-Google-Smtp-Source: AGHT+IE6lmyFVDIzRDjMNMDJr7ziWFO9lcVLRUhmriJlMF/fW0JeEBIeoIvfxe2utlR5wgFziPhLlg== X-Received: by 2002:a05:600c:3485:b0:426:66a0:6df6 with SMTP id 5b1f17b1804b1-42c29f25f8bmr3176475e9.0.1725354037545; Tue, 03 Sep 2024 02:00:37 -0700 (PDT) Received: from google.com (109.36.187.35.bc.googleusercontent.com. [35.187.36.109]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42bb6df84b9sm163907905e9.24.2024.09.03.02.00.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Sep 2024 02:00:37 -0700 (PDT) Date: Tue, 3 Sep 2024 09:00:32 +0000 From: Mostafa Saleh To: Jason Gunthorpe Cc: acpica-devel@lists.linux.dev, Hanjun Guo , iommu@lists.linux.dev, Joerg Roedel , Kevin Tian , kvm@vger.kernel.org, Len Brown , linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Lorenzo Pieralisi , "Rafael J. Wysocki" , Robert Moore , Robin Murphy , Sudeep Holla , Will Deacon , Alex Williamson , Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: Re: [PATCH v2 8/8] iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED Message-ID: References: <0-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com> <8-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com> <20240830170426.GV3773488@nvidia.com> <20240903003022.GF3773488@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240903003022.GF3773488@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240903_020040_675926_2021CB39 X-CRM114-Status: GOOD ( 48.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Sep 02, 2024 at 09:30:22PM -0300, Jason Gunthorpe wrote: > On Mon, Sep 02, 2024 at 09:57:45AM +0000, Mostafa Saleh wrote: > > > > 2) Is there a reason the UAPI is designed this way? > > > > The way I imagined this, is that userspace will pass the pointer to the CD > > > > (+ format) not the STE (or part of it). > > > > > > Yes, we need more information from the STE than just that. EATS and > > > STALL for instance. And the cachability below. Who knows what else in > > > the future. > > > > But for example if that was extended later, how can user space know > > which fields are allowed and which are not? > > Changes the vSTE rules that require userspace being aware would have > to be signaled in the GET_INFO answer. This is the same process no > matter how you encode the STE bits in the structure. > How? And why changing that in the future is not a problem as sanitising IDRs? > This confirmation of kernel support would then be reflected in the > vIDRs to the VM and the VM could know to set the extended bits. > > Otherwise setting an invalidate vSTE will fail the ioctl, the VMM can > log the event, generate an event and install an abort vSTE. > > > > Overall this sort of direct transparency is how I prefer to see these > > > kinds of iommufd HW specific interfaces designed. From a lot of > > > experience here, arbitary marshall/unmarshall is often an > > > antipattern :) > > > > Is there any documentation for the (proposed) SMMUv3 UAPI for IOMMUFD? > > Just the comments in this series? But this is a UAPI. How can userspace implement that if it has no documentation, and how can it be maintained if there is no clear interface with userspace with what is expected/returned... > > > I can understand reading IDRs from userspace (with some sanitation), > > but adding some more logic to map vSTE to STE needs more care of what > > kind of semantics are provided. > > We can enhance the comment if you think it is not clear enough. It > lists the fields the userspace should pass through. > > > Also, I am working on similar interface for pKVM where we “paravirtualize” > > the SMMU access for guests, it’s different semantics, but I hope we can > > align that with IOMMUFD (but it’s nowhere near upstream now) > > Well, if you do paravirt where you just do map/unmap calls to the > hypervisor (ie classic virtio-iommu) then you don't need to do very > much. But we have a different model, with virtio-iommu, it typically presents the device to the VM and on the backend it calls VFIO MAP/UNMAP. Although technically we can have virtio-iommu in the hypervisor (EL2), that is a lot of complexit and increase in the TCB of pKVM. For pKVM, the VMM is not trusted and the hypervisor would do the map/unmap..., but the VMM will have to configure the virtual view of the device (Mapping of endpoints to virtual endpoints, vIRQs…), this requires a userspace interface to query some HW info (similar to VFIO VFIO_DEVICE_GET_IRQ_INFO and then mapping it to a GSI through KVM, but for IOMMUs) Though, this design is very early and in progress. > > If you want to do nesting, then IMHO, just present a real vSMMU. It is > already intended to be paravirtualized and this is what the > confidential compute people are going to be doing as well. > > Otherwise I'd expect you'd get more value to align with the > virtio-iommu nesting stuff, where they have layed out what information > the VM needs. iommufd is not intended to be just jammed directly into > a VM. There is an expectation that a VMM will sit there on top and > massage things. I haven’t been keeping up with iommufd lately, I will try to spend more time on that in the future. But my idea is that we would create an IOMMUFD, attach it to a device and then through some extra IOCTLs, we can configure some “virtual” topology for it which then relies on KVM, again this is very early, and we need to support pKVM IOMMUs in the host first (I plan to send v2 RFC soon for that) > > > I see you are talking in LPC about IOMMUFD: > > https://lore.kernel.org/linux-iommu/0-v1-01fa10580981+1d-iommu_pt_jgg@nvidia.com/T/#m2dbb08f3bf8506a492bc7dda2de662e42371e683 > > > > Do you have any plans to talk about this also? > > Nothing specific, this is LPC so if people in the room would like to > use the session for that then we can talk about it. Last year the room > wanted to talk about PASID mostly. > > I haven't heard if someone is going to KVM forum to talk about > vSMMUv3? Eric? Nicolin do you know? I see, I won’t be in KVM forum, but I plan to attend LPC, we can discuss further there if people are interested. Thanks, Mostafa > > Jason