From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF108C4708F for ; Fri, 4 Jun 2021 03:40:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8ED6B61402 for ; Fri, 4 Jun 2021 03:40:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229751AbhFDDmA (ORCPT ); Thu, 3 Jun 2021 23:42:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44023 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229761AbhFDDmA (ORCPT ); Thu, 3 Jun 2021 23:42:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622778014; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aFUnYpKRreDc48wVwuXCdbokDvF9b+3ny8yIoFA0wCM=; b=amDeYvdCtCn1iUg1XfCfRvzF+WZn1mR2WDOmWc+VGfybhwYMmtskVlp5kw8i5N1WX7Hgdw yKIEATS9jwDKFa5mHOqtk0OCGmLnA1p76Q1SWdDiDTvQbOgKLsGHlErveLnVcSGPSAcNOQ pArJlzk92lTCllTJK/skI2o0vunJUUk= Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-311-WZkuOp5qNOyTEol5bm92mw-1; Thu, 03 Jun 2021 23:40:12 -0400 X-MC-Unique: WZkuOp5qNOyTEol5bm92mw-1 Received: by mail-ot1-f70.google.com with SMTP id i25-20020a9d4a990000b0290304f00e3e3aso4349856otf.15 for ; Thu, 03 Jun 2021 20:40:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aFUnYpKRreDc48wVwuXCdbokDvF9b+3ny8yIoFA0wCM=; b=KAKMb8gTuL1/KUzO5OkF0oOKCn35L7pf5jYyMQoB21eNyBPFsDuwDFR2MV72TWrDBF lpFhen0Hod/hghOIEN5ZW8tP79D/V2MrqzeHLn7FKdmGqplEendEFifS5z8DMJRWac1f JjcXH1fvDpHTCc2ZGWnUV5buTenv5UMZFpCrPDthyZ6sA//vKoakC8gOORfryCIFYsvZ gEvM5ndrWzuUYW7tk0ngLOH4Pw3mVxdyeSc+0pP5x5hr85L3y1wg+YgsnxzZjjzLEXbw Ar1kdFyJjbeAKRUFCjMfZDRQc1UFl7a2qhUQOVst2Y/i/DAhgKG8PE2P949yZkTuwf1v sSuQ== X-Gm-Message-State: AOAM5309bBBe/ORHDkOtNml21KKPKW8vv5Ur9j3n1R4JsCjGTxpfYXGV ntoErNYNB8hxGxHPV8bHcPF0CIQYiWPCyTkshF1fga1nflXiVwYqlXGjzbs/irfY/PGYV8O2Ypn ruznJGRQoaBM7+AeTTlmF X-Received: by 2002:a54:4494:: with SMTP id v20mr1624080oiv.74.1622778012141; Thu, 03 Jun 2021 20:40:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxyoQSP7oUcYQgpMaIhA3jFP+XxgrJ7TW7ji9ff5F2mCl2XeqDgdz5AJt7rZgUmclwKq5uEIg== X-Received: by 2002:a54:4494:: with SMTP id v20mr1624062oiv.74.1622778011907; Thu, 03 Jun 2021 20:40:11 -0700 (PDT) Received: from redhat.com ([198.99.80.109]) by smtp.gmail.com with ESMTPSA id d1sm223468otu.9.2021.06.03.20.40.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Jun 2021 20:40:11 -0700 (PDT) Date: Thu, 3 Jun 2021 21:40:09 -0600 From: Alex Williamson To: "Tian, Kevin" Cc: Jason Gunthorpe , Cornelia Huck , Kirti Wankhede , "Jiang, Dave" , "tglx@linutronix.de" , "vkoul@kernel.org" , "Raj, Ashok" , "Dey, Megha" , "Pan, Jacob jun" , "Liu, Yi L" , "Lu, Baolu" , "Kumar, Sanjay K" , "Luck, Tony" , "Williams, Dan J" , "eric.auger@redhat.com" , "pbonzini@redhat.com" , "dmaengine@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" Subject: Re: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver Message-ID: <20210603214009.68fac0c4.alex.williamson@redhat.com> In-Reply-To: References: <162164243591.261970.3439987543338120797.stgit@djiang5-desk3.ch.intel.com> <20210523232219.GG1002214@nvidia.com> <86cde154-37c7-c00d-b0c6-06b15b50dbf7@intel.com> <20210602231747.GK1002214@nvidia.com> <20210603014932.GN1002214@nvidia.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org On Thu, 3 Jun 2021 05:52:58 +0000 "Tian, Kevin" wrote: > > From: Jason Gunthorpe > > Sent: Thursday, June 3, 2021 9:50 AM > > > > On Thu, Jun 03, 2021 at 01:11:37AM +0000, Tian, Kevin wrote: > > > > > Jason, can you clarify your attitude on mdev guid stuff? Are you > > > completely against it or case-by-case? If the former, this is a big > > > decision thus it's better to have consensus with Alex/Kirti. If the > > > latter, would like to hear your criteria for when it can be used > > > and when not... > > > > I dislike it generally, but it exists so . I know others feel > > more strongly about it being un-kernely and the wrong way to use sysfs. > > > > Here I was remarking how the example in the cover letter made the mdev > > part seem totally pointless. If it is pointless then don't do it. > > Is your point about that as long as a mdev requires pre-config > through driver specific sysfs then it doesn't make sense to use > mdev guid interface anymore? Can you describe exactly what step 1. is doing in this case from the original cover letter ("Enable wq with "mdev" wq type")? That does sound a bit like configuring something to use mdev then separately going to the trouble of creating the mdev. As Jason suggests, if a wq is tagged for mdev/vfio, it could just register itself as a vfio bus driver. But if we want to use mdev, why doesn't available_instances for your mdev type simply report all unassigned wq and the `echo $UUID > create` grabs a wq for mdev? That would remove this pre-config contention, right? > The value of mdev guid interface is providing a vendor-agnostic > interface for mdev life-cycle management which allows one- > enable-fit-all in upper management stack. Requiring vendor > specific pre-config does blur the boundary here. We need to be careful about using work-avoidance in the upper management stack as a primary use case for an interface though. > Alex/Kirt/Cornelia, what about your opinion here? It's better > we can have an consensus on when and where the existing > mdev sysfs could be used, as this will affect every new mdev > implementation from now on. I have a hard time defining some fixed criteria for using mdev. It's essentially always been true that vendors could write their own vfio "bus driver", like vfio-pci or vfio-platform, specific to their device. Mdevs were meant to be a way for the (non-vfio) driver of a device to expose portions of the device through mediation for use with vfio. It seems like that's largely being done here. What I think has changed recently is this desire to make it easier to create those vendor drivers and some promise of making module binding work to avoid the messiness around picking a driver for the device. In the auxiliary bus case that I think Jason is working on, it sounds like the main device driver exposes portions of itself on an auxiliary bus where drivers on that bus can integrate into the vfio subsystem. It starts to get pretty fuzzy with what mdev already does, but it's also a more versatile interface. Is it right for everyone? Probably not. Is the pre-configuration issue here really a push vs pull problem? I can see the requirement in step 1. is dedicating some resources to an mdev use case, so at that point it seems like the argument is whether we should just create aux bus devices that get automatically bound to a vendor vfio-pci variant and we avoid the mdev lifecycle, which is both convenient and ugly. On the other hand, mdev has more of a pull interface, ie. here are a bunch of device types and how many of each we can support, use create to pull what you need. > > Remember we have stripped away the actual special need to use > > mdev. You don't *have* to use mdev anymore to use vfio. That is a > > significant ideology change even from a few months ago. > > > > Yes, "don't have to" but if there is value of doing so it's > not necessary to blocking it? One point in my mind is that if > we should minimize vendor-specific contracts for user to > manage mdev or subdevice... Again, this in itself is not a great justification for using mdev, we're creating vendor specific device types with vendor specific additional features, that could all be done via some sort of netlink interface too. The thing that pushes this more towards mdev for me is that I don't think each of these wqs appear as devices to the host, they're internal resources of the parent device and we want to compose them in ways that are slightly more amenable to traditional mdevs... I think. Thanks, Alex