From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yGcrP0yRjzDqlv for ; Wed, 18 Oct 2017 01:24:04 +1100 (AEDT) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v9HENuej110504 for ; Tue, 17 Oct 2017 10:24:02 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0a-001b2d01.pphosted.com with ESMTP id 2dnk4w1wkr-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 17 Oct 2017 10:24:02 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 17 Oct 2017 10:24:00 -0400 Subject: Re: [PATCH v3 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device To: Bjorn Helgaas , "Bryant G. Ly" Cc: Alex Williamson , Steven Royer , Michael Ellerman , bhelgaas@google.com, benh@kernel.crashing.org, paulus@samba.org, linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, "Juan J . Alvarez" , Bodong Wang , Eli Cohen , Saeed Mahameed References: <20170922141928.49141-3-bryantly@linux.vnet.ibm.com> <20171011200524.GR25517@bhelgaas-glaptop.roam.corp.google.com> <87efq92ei6.fsf@concordia.ellerman.id.au> <20171012182932.GA653@bhelgaas-glaptop.roam.corp.google.com> <20171013033456.GC25517@bhelgaas-glaptop.roam.corp.google.com> <88eab212351a662dda330071d9afe9dc@linux.vnet.ibm.com> <11c15c773aefcc0490e197051c2eaf41@linux.vnet.ibm.com> <20171013120558.1a129183@t450s.home> <24fdec01-8df4-f027-0b3c-0472a2d73946@linux.vnet.ibm.com> <20171017135123.GA5641@bhelgaas-glaptop.roam.corp.google.com> From: Juan Alvarez Date: Tue, 17 Oct 2017 09:23:54 -0500 MIME-Version: 1.0 In-Reply-To: <20171017135123.GA5641@bhelgaas-glaptop.roam.corp.google.com> Content-Type: multipart/alternative; boundary="------------C453CE4B6C2C1635104F5150" Message-Id: List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , This is a multi-part message in MIME format. --------------C453CE4B6C2C1635104F5150 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 10/17/17 8:51 AM, Bjorn Helgaas wrote: > This is where I get confused. I guess the Linux that sets > PCI_SRIOV_CTRL_VFE to enable the VFs can also perform config accesses > to the VFs, since it can enumerate them and build pci_dev structs for > them, right? Correct, we are not changing anything related to how the VF gets initialized in the Linux kernel. > > And the Linux in the "Hosting Partition" is a guest that cannot see a > VF until a management console attaches the VF to the Hosting > Partition? Correct, this is how PHYP does normal adapter assignment. > I'm not a VFIO or KVM expert but that sounds vaguely like > what they would do when assigning a VF to a guest. I do not know the specifics on VFIO and KVM. However, in this case there is no KVM running on the Linux (LPAR) logical partition. PHYP owns all the system resources. > >> So like existing way of enabling SRIOV we still rely on the PF driver to >> enable VFs - but in this case the attachment phase is done via a user >> action via a management console in our case (novalink or hmc) triggered >> event that will essentially act like a hotplug. >> >> So in the fine details of that user triggered action the system >> firmware will bind the VFs, allowing resources to be allocated to >> the VF. - Which essentially does all the attaching as we know it >> today but is managed by PHYP not by the kernel. > What exactly does "firmware binding the VFs" mean? I guess this must > mean assigning a VF to a partition, injecting a hotplug add event to > that partition, and making the VF visible in config space? Firmware basically adds a device node to the device tree as defined in the (PAPR) Power Architecture Platform Requirements document. Juan J. Alvarez --------------C453CE4B6C2C1635104F5150 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit

On 10/17/17 8:51 AM, Bjorn Helgaas wrote:

This is where I get confused.  I guess the Linux that sets
PCI_SRIOV_CTRL_VFE to enable the VFs can also perform config accesses
to the VFs, since it can enumerate them and build pci_dev structs for
them, right?
Correct, we are not changing anything related to how the VF gets initialized
in the Linux kernel. 

And the Linux in the "Hosting Partition" is a guest that cannot see a
VF until a management console attaches the VF to the Hosting
Partition?
Correct, this is how PHYP does normal adapter assignment.

 I'm not a VFIO or KVM expert but that sounds vaguely like
what they would do when assigning a VF to a guest.
I do not know the specifics on VFIO and KVM. However, in this
case there is no KVM running on the Linux (LPAR) logical partition.
PHYP owns all the system resources.


So like existing way of enabling SRIOV we still rely on the PF driver to
enable VFs - but in this case the attachment phase is done via a user
action via a management console in our case (novalink or hmc) triggered
event that will essentially act like a hotplug.

So in the fine details of that user triggered action the system
firmware will bind the VFs, allowing resources to be allocated to
the VF.  - Which essentially does all the attaching as we know it
today but is managed by PHYP not by the kernel.

What exactly does "firmware binding the VFs" mean?  I guess this must
mean assigning a VF to a partition, injecting a hotplug add event to
that partition, and making the VF visible in config space?
Firmware basically adds a device node to the device tree as defined
in the (PAPR) Power Architecture Platform Requirements document.

Juan J. Alvarez
--------------C453CE4B6C2C1635104F5150--