From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83DBDECE564 for ; Tue, 10 Sep 2024 11:13:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UYy4DmMGHFHCTGKn7bORTP8d4lFk3R5EHaNq4YeXwzM=; b=xqsZTdQ9afbzpnQYHCzjFMvHC5 HGb2ivU66trl1M8DrUjT+vUb40I9kS9kd7e+GKUfxvPqRhT3+FuIdOI5//uufiJRlietdPBONpd+7 rc+72mH/tK1XDhZkTg/9XxwuXHyL811up1qCoiv04W6ydQxkdoTWQo8q+48j2JULqxB6MOEev4GB/ o8qMTMtU5FTJ/nsAs2q1klf0dD3TczgrWDt66xAibOZEu75vtLL7PAFsHv5YidORzYNS/tpXKMgLE e5uewjV8rZyxl90brlNJLjHwvmYfSVeCgVaZgVi5snz+7R5RJij2rtorY0YRGoN4ryzgyrQDAM7zI VpklSHjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1snyoO-00000005LkG-2LVu; Tue, 10 Sep 2024 11:13:32 +0000 Received: from mail-ed1-x534.google.com ([2a00:1450:4864:20::534]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1snynL-00000005Lf4-45LI for linux-arm-kernel@lists.infradead.org; Tue, 10 Sep 2024 11:12:29 +0000 Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-5c2460e885dso11994a12.0 for ; Tue, 10 Sep 2024 04:12:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725966746; x=1726571546; darn=lists.infradead.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=UYy4DmMGHFHCTGKn7bORTP8d4lFk3R5EHaNq4YeXwzM=; b=XTE1nZQSceMGs8dv8MtReYnYMyzkjEf1eHmX0rilb1aTQMf7o9cRUtiw2iLVi3Iq8o vmra+1q0p4mHXcNyVCTrbfuU9eGnYKNAiRQHKug9sNCwoeDPYXVHF8LDeALkB77bxg5K 2lYPQmhmlThsCiBmx/AeGPitBqnPtsFPeBuvu+/9c/bHEzjswN/4OTXWKO7n3fJhJJ7b KvB40dw5uaZPey7Jc1SD29ZmNQuOYS+aDA3IHnMsauefk6TXgQuFYEn1iSi9489z2bXF XBYmkQYMQIIKQLOuuK+1VLusYsLXpMQ1nIT930QOKQz+bO0kCtSYw0Q6nvuwkfm49Kze 8JTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725966746; x=1726571546; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UYy4DmMGHFHCTGKn7bORTP8d4lFk3R5EHaNq4YeXwzM=; b=QnjRRQt0pNSH/4diWg5Jvse3eFj2PNvbUTnHd9LdIlh7Y7qRLhtFP3WODSyQI56CIe JIABg0nDP6/e5zWoMV/fOLrG8DAzYMrapV/AmR5Ho4/yjJDSdKFq8fz44tVR26LNWurX AxkwNP6QtzM0nEHhZjfOKccMgwjThX/Q1qAtADB3MIIf/dGQ0qM52IRVg+Khk9GaJy73 Ly4P4op5DpIvLmDzBW76flEZmj1jHCgcRdnnOWijXggkvENHmS9f5styM9klIW3bJhgl 4eLwoijPSKEOZoFYPMGjWdmhga/mxpO89JZEesraJxUXIyySN5Pk6CG4lmAKsYTTbOk3 ynhA== X-Forwarded-Encrypted: i=1; AJvYcCUNYGQBJlT+ZnK6WBbphrzttnzE68U2F9NLlPJaHJlA4R9jPy+N2Ksu33e7p6fQJkwO+8Dct1h2EFLh0olWa77l@lists.infradead.org X-Gm-Message-State: AOJu0Ywsrg6p/u7Ne1vYofA0Wyktdwr1zcrlc/NVoWwKCJ0XBjiNM70l xg2IX9pgZGTgRLbOtNl62pi1H9N2cgMd6zIyEmFNnDbhLZf19PfhniDlm6ch5w== X-Google-Smtp-Source: AGHT+IFie5nPTW6D5tNlRl7qFWvNiI/ySCh0zgRrb+aBAPhZhb7eVrcOK1eXSiw1bugwbN+BJ9aSog== X-Received: by 2002:a05:6402:1ec4:b0:59f:9f59:9b07 with SMTP id 4fb4d7f45d1cf-5c4029ef54amr158266a12.4.1725966745263; Tue, 10 Sep 2024 04:12:25 -0700 (PDT) Received: from google.com (205.215.190.35.bc.googleusercontent.com. [35.190.215.205]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42caeb8120asm107512605e9.37.2024.09.10.04.12.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Sep 2024 04:12:24 -0700 (PDT) Date: Tue, 10 Sep 2024 11:12:20 +0000 From: Mostafa Saleh To: Jason Gunthorpe Cc: acpica-devel@lists.linux.dev, Hanjun Guo , iommu@lists.linux.dev, Joerg Roedel , Kevin Tian , kvm@vger.kernel.org, Len Brown , linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Lorenzo Pieralisi , "Rafael J. Wysocki" , Robert Moore , Robin Murphy , Sudeep Holla , Will Deacon , Alex Williamson , Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: Re: [PATCH v2 8/8] iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED Message-ID: References: <0-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com> <8-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com> <20240830170426.GV3773488@nvidia.com> <20240903003022.GF3773488@nvidia.com> <20240903235532.GJ3773488@nvidia.com> <20240906133444.GE1358970@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240906133444.GE1358970@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240910_041228_038294_0EA37F80 X-CRM114-Status: GOOD ( 44.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Sep 06, 2024 at 10:34:44AM -0300, Jason Gunthorpe wrote: > On Fri, Sep 06, 2024 at 11:07:47AM +0000, Mostafa Saleh wrote: > > > However, I believe the UAPI can be more clear and solid in terms of > > what is supported (maybe a typical struct with the CD, and some > > extra configs?) I will give it a think. > > I don't think breaking up the STE into fields in another struct is > going to be a big improvement, it adds more code and corner cases to > break up and reassemble it. > > #define STRTAB_STE_0_NESTING_ALLOWED \ > cpu_to_le64(STRTAB_STE_0_V | STRTAB_STE_0_CFG | STRTAB_STE_0_S1FMT | \ > STRTAB_STE_0_S1CTXPTR_MASK | STRTAB_STE_0_S1CDMAX) > #define STRTAB_STE_1_NESTING_ALLOWED \ > cpu_to_le64(STRTAB_STE_1_S1DSS | STRTAB_STE_1_S1CIR | \ > STRTAB_STE_1_S1COR | STRTAB_STE_1_S1CSH | \ > STRTAB_STE_1_S1STALLD | STRTAB_STE_1_EATS) > > It is 11 fields that would need to be recoded, that's alot.. Even if > you say the 3 cache ones are not needed it is still alot. I was thinking of providing a higher level semantics (no need for caching, valid...), something like: struct smmu_user_table { u64 cd_table; u32 smmu_cd_cfg; /* linear or 2lvl,.... */ u32 smmu_trans_cfg; /* Translate, bypass, abort */ u32 dev_feat; /*ATS, STALL, …*/ }; I feel that is a bit more clear for user space? Instead of partially setting the STE, and it should be easier to extend than masking the STE. I’am not opposed to the vSTE, I just feel it's loosely defined, that's why I was asking for the docs. > > > > Reporting a static kernel capability through GET_INFO output is > > > easier/saner than providing some kind of policy flags in the GET_INFO > > > input to specify how the sanitization should work. > > > > I don’t think it’s “policy”, it’s just giving userspace the minimum > > knowledge it needs to create the vSMMU, but again no really strong > > opinion about that. > > There is no single "minimum knowledge" though, it depends on what the > VMM is able to support. IMHO once you go over to the "VMM has to > ignore bits it doesn't understand" you may as well just show > everything. Then the kernel side can't be wrong. > > If the kernel side can be wrong, then you are back to handshaking > policy because the kernel can't assume that all existing VMMs wil not > rely on the kernel to do the masking. > I agree it’s tricky, again no strong opinion on that, although I doubt that a VMM would care about all the SMMU features. > > > > But this is a UAPI. How can userspace implement that if it has no > > > > documentation, and how can it be maintained if there is no clear > > > > interface with userspace with what is expected/returned... > > > > > > I'm not sure what you are looking for here? I don't think an entire > > > tutorial on how to build a paravirtualized vSMMU is appropriate to > > > put in comments? > > > > Sorry, I don’t think I was clear, I meant actual documentation for > > the UAPI, as in RST files for example. If I want to support that > > in kvmtool how can I implement it? > > Well, you need thousands of lines of code in kvtool to build a vIOMMU :) > > Nicolin is looking at writing something, lets see. > > I think for here we should focus on the comments being succinct but > sufficient to understand what the uAPI does itself. > Actually I think the opposite, I think UAPI docs is more important here, especially for the vSTE, that's how we can compare the code to what is expected from user-space. > > > I would *really* like everyone to sit down and figure out how to > > > manage virtual device lifecycle in a single language! > > > > Yes, just like the guest_memfd work. There has been also > > some work to unify some of the guest HVC bits: > > https://lore.kernel.org/all/20240830130150.8568-1-will@kernel.org/ > > I think Dan Williams is being ringleader for the PCI side effort on CC Thanks, I will try to spend some time on the secure VFIO work. Thanks, Mostafa > > Jason