From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38EB6C433EF for ; Thu, 7 Apr 2022 19:08:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B337140606; Thu, 7 Apr 2022 19:08:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QJcz10EpDRFY; Thu, 7 Apr 2022 19:08:40 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 69CB6400FE; Thu, 7 Apr 2022 19:08:40 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3E6A1C001D; Thu, 7 Apr 2022 19:08:40 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 52D66C0012 for ; Thu, 7 Apr 2022 19:08:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2CCD681420 for ; Thu, 7 Apr 2022 19:08:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=nvidia.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6OBktBDRB9ys for ; Thu, 7 Apr 2022 19:08:33 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2062d.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e8a::62d]) by smtp1.osuosl.org (Postfix) with ESMTPS id 91254813AE for ; Thu, 7 Apr 2022 19:08:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fVikd7plE6MvNlYXMqjrStMcN63MeFfjFkh8U0BaYEK0RXQYOmAmb5fos7gy8r88J6pgwbwIzg+W1GfaNM1Jv/R8GW/UBNYzCtHoWo5Cj9xCiZc/C+MwbJxI1WQzyilCDeWcrFtehNz23Z08MlhowacGMaaPyQ7td+Cnj9DJ/JbOgkr2Uikn7GXPBFR99WuJxQxP3uVHO22XUQCbIprMVm2hB1QpmC8tj6OtqVOOQMXU1mqA4Eq9rGzGfgSSNvGxnSyaV7W8AQhrw9/wHce82S5muJP+us6+ZyQlEziiHrfqiriR1E44NTr/Pdv3xYuY5G7E1kpZQFRVkA6OiwrwPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jxASJizsrCn9W6xNht5bqMVXF7gC3fNcbRzt0jzeLwg=; b=Tmr09toUZ3D9qoq0K+El+Ni+HKGS1IEOwtGtc0jV1sOaThCqSd74gNXBn1wsam781/tMezkxufbCuUqbojhQLj7kd0xdTokUCgP9YTqANObPAti0oLyxEgZPy3Xc4xSoyser/ir+HbjiDt0J0bwPUmNJJ5N1wehYYWYKGJ4qMvhq/urf98x7dmvE4FsEK4aM6wjPZQZgcp5zTOlPQBYshqGsaKjlouIi5suIinOGMU9h0xHk40XjRHeRx/wLImcmTZngMzdbPz/ehlKxqKWwXIghP3TKAY4pgoE23Cp9hdnpVFUbd9rvTe8iKNcsqtF5oLk21Up8X6FQNGt+ti2EXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jxASJizsrCn9W6xNht5bqMVXF7gC3fNcbRzt0jzeLwg=; b=jn/QUgJJ/duGEjtEuWWu+sKoyRWwwiFipOZDPmHDSowm2+MwqfhAXJ0/RW3FXdEGHL0qI+R1XSAr7tmcvDRqT/kcYj9dsRNFm8NfkimmUBNTkx8HRziQs03YhRR6rbbmyS976nrZg3dqOfwePWy8WYm0tQDY6Ie6GCAJ3NL4OwnTyVwcHy9eSjbLtg4cCVnTXNbiuUj4nzKjscJPIw441bFIOawB4SCcXX974jroLyqo9IB93RrKan7V0EufETP1FjLtks77sgUv++yKImRrXgaBb2kpeApkuEVT/lbdJDnqIr70NCKHCvAYfFLdpqUh+JlM0UZPbirvagMmoCl/fQ== Received: from BY5PR12MB3841.namprd12.prod.outlook.com (2603:10b6:a03:194::27) by MN0PR12MB5956.namprd12.prod.outlook.com (2603:10b6:208:37f::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.19; Thu, 7 Apr 2022 19:08:30 +0000 Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BY5PR12MB3841.namprd12.prod.outlook.com (2603:10b6:a03:194::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.21; Thu, 7 Apr 2022 19:08:28 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::cdfb:f88e:410b:9374]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::cdfb:f88e:410b:9374%6]) with mapi id 15.20.5144.022; Thu, 7 Apr 2022 19:08:28 +0000 Date: Thu, 7 Apr 2022 16:08:24 -0300 To: Robin Murphy Subject: Re: [PATCH v2 0/4] Make the iommu driver no-snoop block feature consistent Message-ID: <20220407190824.GS2120790@nvidia.com> References: <0-v2-f090ae795824+6ad-intel_no_snoop_jgg@nvidia.com> <20220407174326.GR2120790@nvidia.com> <77482321-2e39-fc7c-09b6-e929a851a80f@arm.com> Content-Disposition: inline In-Reply-To: <77482321-2e39-fc7c-09b6-e929a851a80f@arm.com> X-ClientProxiedBy: BY3PR10CA0025.namprd10.prod.outlook.com (2603:10b6:a03:255::30) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1549235e-ceeb-4e63-70f5-08da18c9ffd0 X-MS-TrafficTypeDiagnostic: BY5PR12MB3841:EE_|MN0PR12MB5956:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uz7xymfURypiDEndU4fk4cW85zkiD7TrM3gfm/qtEgxGVBgEa5t+1oc0cWyqRqeswRmRLPqMRCn2L8zJ3XJYfomzPCQnH6IHxj/lVdd94zBo5K3n6OUZ8ZqTSWQ5ayzEhP0dU5cDHzxy3oSxzcuf+NbI7cLGdwwi+jEPFB97fUYQ5iA9wjlnDE6NgUQ4oLZ5eJsi4ujLab5V/4K5AG0mJl4sHh9r88hZMCoBRlEQ+EccZhqAJkMz1J8KfGYuRStLmx1IyJi6FqCdrEytGfJZ16I5fJljXeunc41BE0wJDS4UWItacgfr7ecnp5hwxjmpxAKJKdAM0BF2N9kDLmyOzt+fi8jW58P3kPbGjMu+0bWyk4bGLrEIBY1zZmRonz8XTjd6hrnQV8MTj0LaTY0uoyqfGLCO8LRQRb67/X6/JtepRtelLup+84MS7ibAuOaJ/yEznLDyM4Yb+VKqjtwfLA9x+HIpAJn7V4xEMHR8hy6m2gc+k1dnHkO0k2p4g8aB0H9vrGdvIGbIrN4EOqkB9qse0r0CCQtiFt6hMnavuxJz12fHfU9dh6S7yFFKNyx79Qx6zrA9wqBKwPoWqylydHDzuBU5WJj0O0IhUDgY+iR68tYNp4VZPuGMBhpRUwUCMbrLF1wP11o+YS60EUXouA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BY5PR12MB3841.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(54906003)(36756003)(8936002)(6512007)(316002)(186003)(6666004)(6916009)(86362001)(26005)(83380400001)(66556008)(2616005)(5660300002)(508600001)(1076003)(38100700002)(33656002)(66946007)(2906002)(4326008)(66476007)(8676002)(7416002)(53546011)(6506007)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?D4RMAht0L3exhw4WcTLsbHTqUgrH8lVFusIlIbHNz67DDoQNewhdUGGRwtYe?= =?us-ascii?Q?mO9eVxGb/aYgJkeMpl3M/5v8Ry5AOdqTjbMZpJKKfNZ9RhkT3zRLC3RDLWfg?= =?us-ascii?Q?xFbjAl5n7Nh1OGchgGiA71AFSHHKAVeOyXAwpjYzOKt9LeEveVZ580SOORbo?= =?us-ascii?Q?sV1UhnhnoEJ6hl1QZrKkpnNkycY/jMGsoyzEDIQMj9uDBbp3gRNG4j5NKLLS?= =?us-ascii?Q?WLv3pbNkQC6KNVVkTE3/xPORs4UNY9y/34O5/AcIZLtILjo2I9r0BUJA3lHJ?= =?us-ascii?Q?i52wH+ctviksipuMF+LUUiUc9VTVOt1YKfR1gX35WPtT8ILrqB7fojBRs27E?= =?us-ascii?Q?pexu4D/f8a7RageUf7pN3od4zw8knkOKVkBdtk41FU39WPjr8XPPSxgQJaD8?= =?us-ascii?Q?1pXui3FButEEHHnnmXia45O3Rh69/uiQ5liuMrMw+cv7rMI5r4nl2ffiiKnD?= =?us-ascii?Q?F+Ki5flPTAKRoV0K6oo4jOVC4WLq8zNq6Yop3Cxgsy/dUBRQHYts4Uwlj1Cv?= =?us-ascii?Q?SX0pBiGu/GfsL4iT+8yJHOCHuHjhytRSVUVu7g5CxpEGbWB0s0SlBWdxNT3/?= =?us-ascii?Q?yLXJ6OUp3DZoBmBG7TjwtXydv9tNlnhZfuVDhnclOvgZYKq1NtuVyES/i5PY?= =?us-ascii?Q?6DycmR5/aa7yR/QfPI8Fz6Oj5Z36CavUJvXkUnX4qiBrssBVlgh2t4tLmLk6?= =?us-ascii?Q?/RL2X4qDDumVUCHQGWLMyCbDdSq38DY9JqI12FZjbmi0J9GWGvhpfb+vJftf?= =?us-ascii?Q?KPGcWT1NMpv0Y9OyKN9EiKE5OO8lBPJiPl3V6I41G5KFhjORLLSSqYKzGJ8m?= =?us-ascii?Q?bPMCdDHDJOyYa+aOeLw2J/5WauRUO5GZl6PWiWRmkxhPLzkOcfJDN7D/eA3/?= =?us-ascii?Q?7VPjUorsxASdp8JWk8QeQd8Fzc1s7k6A5+zxdxh9YPgM0bzm49glJ73H1cL6?= =?us-ascii?Q?SJMB+Q6kb8GdJRsf1LIgmWFPOPW4zmTpQIsGZzLH54gg9uNLOJ/EaGQqkXEF?= =?us-ascii?Q?XUPLBNEegJkKpmzkna45uTftSmj4FLxJzpCT3KU2cuiaEvJXGJw93asYVSkN?= =?us-ascii?Q?c3pP1yYWCYnQzGMW/bUElkV9Q9XFCpEJVsz7aBa+p9/ujXqh9xikafuBDADO?= =?us-ascii?Q?mi6sjRak/jf7CT52URNne0LWILj6q5wDXwi4QCc1Ji25ilQZnyv3WFAn0Edf?= =?us-ascii?Q?x+jYTTYR3p3BRKiG55O4U3N8ECI4G4SFsE553WKtdcLTxDxTcH8kpDpIf5yD?= =?us-ascii?Q?SOfVTzsVLqjuQFBHm9Ke8UUP+XTnBUi8kEORlrtMW8GQCsUQByYX9rNzlOlM?= =?us-ascii?Q?YgRjwyTvQwhXINmCxqsHy7wcGrfRJMBDtedU9SiPmWSrsw4SplifCJngo7wQ?= =?us-ascii?Q?i6+qmhYr1VtoVRa4KGZzb0tn4eewtZI7oDobbWJhRZJZwgaVXo44KMKJ9bMh?= =?us-ascii?Q?u69Qr5O6f42+cqAPTgt2NOXvSJqBFO3UZtdFXD6fN3Ilub/olfyvW00BBDGn?= =?us-ascii?Q?nNBfo/SKrGTqaolR/wbjy+BiOAgXBqMrMnRr58F9uhbK2RU5p28i+TM9wtrf?= =?us-ascii?Q?ay+GWNm4b2G8FWMJ5UGfhUJCDRQBh+V2jIr2wvDYyogfE48BWkAjONEAA/yj?= =?us-ascii?Q?Dj+oFh5u8u1gJK89KICGGqu0DszzsSMZF78I6Qll3cS+sxObsIdTBSQKq6UF?= =?us-ascii?Q?5JZL5PEqRA+5YFeUY2/4WAvCS9iDymjgOjDjhR7GZI5BgiQB8UqWyMmQ2d8O?= =?us-ascii?Q?8A84WbFpWw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1549235e-ceeb-4e63-70f5-08da18c9ffd0 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2022 19:08:28.6627 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: W7VyaW2mV+Hc005Wolva5cmYsm8WOHZMLKPKLRkVMKAg/MJV8znuxDxK9cTCg0G1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5956 Cc: "Tian, Kevin" , kvm@vger.kernel.org, Will Deacon , Cornelia Huck , iommu@lists.linux-foundation.org, Alex Williamson , David Woodhouse , Christoph Hellwig X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Jason Gunthorpe via iommu Reply-To: Jason Gunthorpe Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Thu, Apr 07, 2022 at 07:02:03PM +0100, Robin Murphy wrote: > On 2022-04-07 18:43, Jason Gunthorpe wrote: > > On Thu, Apr 07, 2022 at 06:03:37PM +0100, Robin Murphy wrote: > > > At a glance, this all looks about the right shape to me now, thanks! > > > > Thanks! > > > > > Ideally I'd hope patch #4 could go straight to device_iommu_capable() from > > > my Thunderbolt series, but we can figure that out in a couple of weeks once > > > > Yes, this does helps that because now the only iommu_capable call is > > in a context where a device is available :) > > Derp, of course I have *two* VFIO patches waiting, the other one touching > the iommu_capable() calls (there's still IOMMU_CAP_INTR_REMAP, which, much > as I hate it and would love to boot all that stuff over to > drivers/irqchip, Oh me too... > it's not in my way so I'm leaving it be for now). I'll have to rebase that > anyway, so merging this as-is is absolutely fine! This might help your effort - after this series and this below there are no 'bus' users of iommu_capable left at all. >From 55d72be40bc0a031711e318c8dd1cb60673d9eca Mon Sep 17 00:00:00 2001 From: Jason Gunthorpe Date: Thu, 7 Apr 2022 16:00:50 -0300 Subject: [PATCH] vfio: Move the IOMMU_CAP_INTR_REMAP to a context with a struct device This check is done to ensure that the device we want to use is fully isolated and the platform does not allow the device's MemWr TLPs to trigger unauthorized MSIs. Instead of doing it in the container context where we only have a group, move the check to open_device where we actually know the device. This is still security safe as userspace cannot trigger an MemWr TLPs without obtaining a device fd. Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 9 +++++++++ drivers/vfio/vfio.h | 1 + drivers/vfio/vfio_iommu_type1.c | 28 +++++++++++++++++----------- 3 files changed, 27 insertions(+), 11 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 9edad767cfdad3..8db5cea1dc1d75 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1355,6 +1355,15 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) if (IS_ERR(device)) return PTR_ERR(device); + /* Confirm this device is compatible with the container */ + if (group->type == VFIO_IOMMU && + group->container->iommu_driver->ops->device_ok) { + ret = group->container->iommu_driver->ops->device_ok( + group->container->iommu_data, device->dev); + if (ret) + goto err_device_put; + } + if (!try_module_get(device->dev->driver->owner)) { ret = -ENODEV; goto err_device_put; diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index a6713022115155..3db60de71d42eb 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -66,6 +66,7 @@ struct vfio_iommu_driver_ops { struct iommu_group *group); void (*notify)(void *iommu_data, enum vfio_iommu_notify_type event); + int (*device_ok)(void *iommu_data, struct device *device); }; int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops); diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index c13b9290e35759..5e966fb0ab9202 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -2153,6 +2153,21 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu, list_splice_tail(iova_copy, iova); } +static int vfio_iommu_device_ok(void *iommu_data, struct device *device) +{ + bool msi_remap; + + msi_remap = irq_domain_check_msi_remap() || + iommu_capable(device->bus, IOMMU_CAP_INTR_REMAP); + + if (!allow_unsafe_interrupts && !msi_remap) { + pr_warn("%s: No interrupt remapping support. Use the module param \"allow_unsafe_interrupts\" to enable VFIO IOMMU support on this platform\n", + __func__); + return -EPERM; + } + return 0; +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group, enum vfio_group_type type) { @@ -2160,7 +2175,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, struct vfio_iommu_group *group; struct vfio_domain *domain, *d; struct bus_type *bus = NULL; - bool resv_msi, msi_remap; + bool resv_msi; phys_addr_t resv_msi_base = 0; struct iommu_domain_geometry *geo; LIST_HEAD(iova_copy); @@ -2257,16 +2272,6 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, INIT_LIST_HEAD(&domain->group_list); list_add(&group->next, &domain->group_list); - msi_remap = irq_domain_check_msi_remap() || - iommu_capable(bus, IOMMU_CAP_INTR_REMAP); - - if (!allow_unsafe_interrupts && !msi_remap) { - pr_warn("%s: No interrupt remapping support. Use the module param \"allow_unsafe_interrupts\" to enable VFIO IOMMU support on this platform\n", - __func__); - ret = -EPERM; - goto out_detach; - } - /* * If the IOMMU can block non-coherent operations (ie PCIe TLPs with * no-snoop set) then VFIO always turns this feature on because on Intel @@ -3159,6 +3164,7 @@ static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = { .open = vfio_iommu_type1_open, .release = vfio_iommu_type1_release, .ioctl = vfio_iommu_type1_ioctl, + .device_ok = vfio_iommu_device_ok, .attach_group = vfio_iommu_type1_attach_group, .detach_group = vfio_iommu_type1_detach_group, .pin_pages = vfio_iommu_type1_pin_pages, -- 2.35.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu