[PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Sathya Perla]

A user must not be allowed to disable VFs while they are already assigned to
a guest. This check is being made in each individual driver that implements
the sriov_configure PCI method.
This patch-set fixes this code duplication by moving this check from
drivers to the sriov_nuvfs_store() routine just before invoking
sriov_configure() when num_vfs is equal to 0.

Vasundhara Volam (4):
  pci: move pci_assivned_vfs() check while disabling VFs to pci
    sub-system
  bnx2x: remove pci_assigned_vfs() check while disabling VFs
  i40e: remove pci_assigned_vfs() check while disabling VFs
  qlcnic: remove pci_assigned_vfs() check while disabling VFs

 drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
 .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
 drivers/pci/pci-sysfs.c                            |    5 +++++
 4 files changed, 7 insertions(+), 17 deletions(-)

[PATCH 1/4] pci: move pci_assivned_vfs() check while disabling VFs to pci sub-system

[Sathya Perla]

From: Vasundhara Volam <vasundhara.volam@emulex.com>

A user must not be allowed to disable VFs while they are already assigned to
a guest. This check is being made in each individual driver that implements
the sriov_configure PCI method.
This patch fixes this code duplication by moving this check to the
sriov_nuvfs_store() routine just before invoking sriov_configure() when
num_vfs is equal to 0.

Signed-off-by: Vasundhara Volam <vasundhara.volam@emulex.com>
Signed-off-by: Sathya Perla <sathya.perla@emulex.com>
---
 drivers/pci/pci-sysfs.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 2c6643f..6e65b47 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -477,6 +477,11 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 	}
 
 	if (num_vfs == 0) {
+		if (pci_vfs_assigned(pdev)) {
+			dev_warn(&pdev->dev, "Cannot disable VFs while they are assigned\n");
+			return -EBUSY;
+		}
+
 		/* disable VFs */
 		ret = pdev->driver->sriov_configure(pdev, 0);
 		if (ret < 0)
-- 
1.7.1

[PATCH 2/4] bnx2x: remove pci_assigned_vfs() check while disabling VFs

[Sathya Perla]

From: Vasundhara Volam <vasundhara.volam@emulex.com>

The pci_assigned_vfs() check (while disabling VFs) is being moved to the
pci-sysfs.c file and will be done before invoking sriov_configure().

Signed-off-by: Vasundhara Volam <vasundhara.volam@emulex.com>
Signed-off-by: Sathya Perla <sathya.perla@emulex.com>
---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
index c88b20a..e90a10b 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
@@ -2481,7 +2481,7 @@ int bnx2x_sriov_configure(struct pci_dev *dev, int num_vfs_param)
 	bp->requested_nr_virtfn = num_vfs_param;
 	if (num_vfs_param == 0) {
 		bnx2x_set_pf_tx_switching(bp, false);
-		bnx2x_disable_sriov(bp);
+		pci_disable_sriov(bp->pdev);
 		return 0;
 	} else {
 		return bnx2x_enable_sriov(bp);
-- 
1.7.1

[PATCH 3/4] i40e: remove pci_assigned_vfs() check while disabling VFs

[Sathya Perla]

From: Vasundhara Volam <vasundhara.volam@emulex.com>

The pci_assigned_vfs() check (while disabling VFs) is being moved to the
pci-sysfs.c file and will be done before invoking sriov_configure().

Signed-off-by: Vasundhara Volam <vasundhara.volam@emulex.com>
Signed-off-by: Sathya Perla <sathya.perla@emulex.com>
---
 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index fff3c27..0028a9a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -951,12 +951,7 @@ int i40e_pci_sriov_configure(struct pci_dev *pdev, int num_vfs)
 	if (num_vfs)
 		return i40e_pci_sriov_enable(pdev, num_vfs);
 
-	if (!pci_vfs_assigned(pf->pdev)) {
-		i40e_free_vfs(pf);
-	} else {
-		dev_warn(&pdev->dev, "Unable to free VFs because some are assigned to VMs.\n");
-		return -EINVAL;
-	}
+	i40e_free_vfs(pf);
 	return 0;
 }
 
-- 
1.7.1

Re: [PATCH 3/4] i40e: remove pci_assigned_vfs() check while disabling VFs

[Jeff Kirsher]

On Mon, Nov 10, 2014 at 3:53 AM, Sathya Perla <sathya.perla@emulex.com> wrote:
> From: Vasundhara Volam <vasundhara.volam@emulex.com>
>
> The pci_assigned_vfs() check (while disabling VFs) is being moved to the
> pci-sysfs.c file and will be done before invoking sriov_configure().
>
> Signed-off-by: Vasundhara Volam <vasundhara.volam@emulex.com>
> Signed-off-by: Sathya Perla <sathya.perla@emulex.com>
> ---
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
>  1 files changed, 1 insertions(+), 6 deletions(-)

Thanks I will add your patch to my queue.

[PATCH 4/4] qlcnic: remove pci_assigned_vfs() check while disabling VFs

[Sathya Perla]

From: Vasundhara Volam <vasundhara.volam@emulex.com>

The pci_assigned_vfs() check (while disabling VFs) is being moved to the
pci-sysfs.c file and will be done before invoking sriov_configure().

Signed-off-by: Vasundhara Volam <vasundhara.volam@emulex.com>
Signed-off-by: Sathya Perla <sathya.perla@emulex.com>
---
 .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
 1 files changed, 0 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c
index a29538b..9802914 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c
@@ -465,16 +465,6 @@ static int qlcnic_pci_sriov_disable(struct qlcnic_adapter *adapter)
 {
 	struct net_device *netdev = adapter->netdev;
 
-	if (pci_vfs_assigned(adapter->pdev)) {
-		netdev_err(adapter->netdev,
-			   "SR-IOV VFs belonging to port %d are assigned to VMs. SR-IOV can not be disabled on this port\n",
-			   adapter->portnum);
-		netdev_info(adapter->netdev,
-			    "Please detach SR-IOV VFs belonging to port %d from VMs, and then try to disable SR-IOV on this port\n",
-			    adapter->portnum);
-		return -EPERM;
-	}
-
 	qlcnic_sriov_pf_disable(adapter);
 
 	rtnl_lock();
-- 
1.7.1

Re: [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Don Dutile]

On 11/10/2014 06:53 AM, Sathya Perla wrote:
> A user must not be allowed to disable VFs while they are already assigned to
> a guest. This check is being made in each individual driver that implements
> the sriov_configure PCI method.
> This patch-set fixes this code duplication by moving this check from
> drivers to the sriov_nuvfs_store() routine just before invoking
> sriov_configure() when num_vfs is equal to 0.
>
> Vasundhara Volam (4):
>    pci: move pci_assivned_vfs() check while disabling VFs to pci
>      sub-system
>    bnx2x: remove pci_assigned_vfs() check while disabling VFs
>    i40e: remove pci_assigned_vfs() check while disabling VFs
>    qlcnic: remove pci_assigned_vfs() check while disabling VFs
>
>   drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
>   drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
>   .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
>   drivers/pci/pci-sysfs.c                            |    5 +++++
>   4 files changed, 7 insertions(+), 17 deletions(-)
>
I have had a side conversation with Alex Williamson, VFIO author.

VFIO is the upstream method that device-assignment is managed/handled on kvm now.
It does not set the PCI_DEV_FLAGS_ASSIGNED pci dev-flags, and thus,
this check will not work when VFIO is used.

This patch set will only work for the former, kvm-managed, device-assignment method,
which is currently being deprecated in qemu as well.

So, yes, it works for kvm managed device-assignment, but not the
newer, VFIO-based device-assignment.

Note, also, that the pci_assigned_vfs() check in the drivers will
always return 0 when VFIO is used for device assignment, so keeping
these checks in the drivers doesn't do what they imply either.

So, taking in the patch solves old, kvm-managed, device assignment,
but a new method is needed when VFIO is involved.

- Don

ps -- Note: just adding the flag setting in vfio-pci does not necessarily
       solve this problem.  VFIO does not know if a device is assigned to a guest;
       it only knows a caller of the ioctl requesting the device to be assigned
       to vfio, and to be dma-mapped for a region of memory, has been requested.
       So, a new PF<->VF mechanism needs to be put in place to
       determine the equivalent information.

Re: [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Alex Williamson]

On Tue, 2014-11-11 at 14:09 -0500, Don Dutile wrote:
> On 11/10/2014 06:53 AM, Sathya Perla wrote:
> > A user must not be allowed to disable VFs while they are already assigned to
> > a guest. This check is being made in each individual driver that implements
> > the sriov_configure PCI method.
> > This patch-set fixes this code duplication by moving this check from
> > drivers to the sriov_nuvfs_store() routine just before invoking
> > sriov_configure() when num_vfs is equal to 0.
> >
> > Vasundhara Volam (4):
> >    pci: move pci_assivned_vfs() check while disabling VFs to pci
> >      sub-system
> >    bnx2x: remove pci_assigned_vfs() check while disabling VFs
> >    i40e: remove pci_assigned_vfs() check while disabling VFs
> >    qlcnic: remove pci_assigned_vfs() check while disabling VFs
> >
> >   drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
> >   drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
> >   .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
> >   drivers/pci/pci-sysfs.c                            |    5 +++++
> >   4 files changed, 7 insertions(+), 17 deletions(-)
> >
> I have had a side conversation with Alex Williamson, VFIO author.
> 
> VFIO is the upstream method that device-assignment is managed/handled on kvm now.
> It does not set the PCI_DEV_FLAGS_ASSIGNED pci dev-flags, and thus,
> this check will not work when VFIO is used.
> 
> This patch set will only work for the former, kvm-managed, device-assignment method,
> which is currently being deprecated in qemu as well.
> 
> So, yes, it works for kvm managed device-assignment, but not the
> newer, VFIO-based device-assignment.
> 
> Note, also, that the pci_assigned_vfs() check in the drivers will
> always return 0 when VFIO is used for device assignment, so keeping
> these checks in the drivers doesn't do what they imply either.
> 
> So, taking in the patch solves old, kvm-managed, device assignment,
> but a new method is needed when VFIO is involved.
> 
> - Don
> 
> ps -- Note: just adding the flag setting in vfio-pci does not necessarily
>        solve this problem.  VFIO does not know if a device is assigned to a guest;
>        it only knows a caller of the ioctl requesting the device to be assigned
>        to vfio, and to be dma-mapped for a region of memory, has been requested.
>        So, a new PF<->VF mechanism needs to be put in place to
>        determine the equivalent information.

pps -- Note: testing pci_assivned_vfs() is racy, nothing prevents the flag
	being added to a device between your check and removing the VF device.
	This is one of the reasons that vfio-pci doesn't use it and that this
	interface should be discouraged in the kernel.

RE: [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Sathya Perla]

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBBbGV4IFdpbGxpYW1zb24gW21h
aWx0bzphbGV4LndpbGxpYW1zb25AcmVkaGF0LmNvbV0NCj4gDQo+IE9uIFR1ZSwgMjAxNC0xMS0x
MSBhdCAxNDowOSAtMDUwMCwgRG9uIER1dGlsZSB3cm90ZToNCj4gPiBPbiAxMS8xMC8yMDE0IDA2
OjUzIEFNLCBTYXRoeWEgUGVybGEgd3JvdGU6DQo+ID4gPiBBIHVzZXIgbXVzdCBub3QgYmUgYWxs
b3dlZCB0byBkaXNhYmxlIFZGcyB3aGlsZSB0aGV5IGFyZSBhbHJlYWR5IGFzc2lnbmVkDQo+IHRv
DQo+ID4gPiBhIGd1ZXN0LiBUaGlzIGNoZWNrIGlzIGJlaW5nIG1hZGUgaW4gZWFjaCBpbmRpdmlk
dWFsIGRyaXZlciB0aGF0DQo+IGltcGxlbWVudHMNCj4gPiA+IHRoZSBzcmlvdl9jb25maWd1cmUg
UENJIG1ldGhvZC4NCj4gPiA+IFRoaXMgcGF0Y2gtc2V0IGZpeGVzIHRoaXMgY29kZSBkdXBsaWNh
dGlvbiBieSBtb3ZpbmcgdGhpcyBjaGVjayBmcm9tDQo+ID4gPiBkcml2ZXJzIHRvIHRoZSBzcmlv
dl9udXZmc19zdG9yZSgpIHJvdXRpbmUganVzdCBiZWZvcmUgaW52b2tpbmcNCj4gPiA+IHNyaW92
X2NvbmZpZ3VyZSgpIHdoZW4gbnVtX3ZmcyBpcyBlcXVhbCB0byAwLg0KPiA+ID4NCj4gPiA+IFZh
c3VuZGhhcmEgVm9sYW0gKDQpOg0KPiA+ID4gICAgcGNpOiBtb3ZlIHBjaV9hc3Npdm5lZF92ZnMo
KSBjaGVjayB3aGlsZSBkaXNhYmxpbmcgVkZzIHRvIHBjaQ0KPiA+ID4gICAgICBzdWItc3lzdGVt
DQo+ID4gPiAgICBibngyeDogcmVtb3ZlIHBjaV9hc3NpZ25lZF92ZnMoKSBjaGVjayB3aGlsZSBk
aXNhYmxpbmcgVkZzDQo+ID4gPiAgICBpNDBlOiByZW1vdmUgcGNpX2Fzc2lnbmVkX3ZmcygpIGNo
ZWNrIHdoaWxlIGRpc2FibGluZyBWRnMNCj4gPiA+ICAgIHFsY25pYzogcmVtb3ZlIHBjaV9hc3Np
Z25lZF92ZnMoKSBjaGVjayB3aGlsZSBkaXNhYmxpbmcgVkZzDQo+ID4gPg0KPiA+ID4gICBkcml2
ZXJzL25ldC9ldGhlcm5ldC9icm9hZGNvbS9ibngyeC9ibngyeF9zcmlvdi5jICB8ICAgIDIgKy0N
Cj4gPiA+ICAgZHJpdmVycy9uZXQvZXRoZXJuZXQvaW50ZWwvaTQwZS9pNDBlX3ZpcnRjaG5sX3Bm
LmMgfCAgICA3ICstLS0tLS0NCj4gPiA+ICAgLi4uL25ldC9ldGhlcm5ldC9xbG9naWMvcWxjbmlj
L3FsY25pY19zcmlvdl9wZi5jICAgfCAgIDEwIC0tLS0tLS0tLS0NCj4gPiA+ICAgZHJpdmVycy9w
Y2kvcGNpLXN5c2ZzLmMgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICA1ICsrKysrDQo+
ID4gPiAgIDQgZmlsZXMgY2hhbmdlZCwgNyBpbnNlcnRpb25zKCspLCAxNyBkZWxldGlvbnMoLSkN
Cj4gPiA+DQo+ID4gSSBoYXZlIGhhZCBhIHNpZGUgY29udmVyc2F0aW9uIHdpdGggQWxleCBXaWxs
aWFtc29uLCBWRklPIGF1dGhvci4NCj4gPg0KPiA+IFZGSU8gaXMgdGhlIHVwc3RyZWFtIG1ldGhv
ZCB0aGF0IGRldmljZS1hc3NpZ25tZW50IGlzIG1hbmFnZWQvaGFuZGxlZA0KPiBvbiBrdm0gbm93
Lg0KPiA+IEl0IGRvZXMgbm90IHNldCB0aGUgUENJX0RFVl9GTEFHU19BU1NJR05FRCBwY2kgZGV2
LWZsYWdzLCBhbmQgdGh1cywNCj4gPiB0aGlzIGNoZWNrIHdpbGwgbm90IHdvcmsgd2hlbiBWRklP
IGlzIHVzZWQuDQo+ID4NCj4gPiBUaGlzIHBhdGNoIHNldCB3aWxsIG9ubHkgd29yayBmb3IgdGhl
IGZvcm1lciwga3ZtLW1hbmFnZWQsIGRldmljZS0NCj4gYXNzaWdubWVudCBtZXRob2QsDQo+ID4g
d2hpY2ggaXMgY3VycmVudGx5IGJlaW5nIGRlcHJlY2F0ZWQgaW4gcWVtdSBhcyB3ZWxsLg0KPiA+
DQo+ID4gU28sIHllcywgaXQgd29ya3MgZm9yIGt2bSBtYW5hZ2VkIGRldmljZS1hc3NpZ25tZW50
LCBidXQgbm90IHRoZQ0KPiA+IG5ld2VyLCBWRklPLWJhc2VkIGRldmljZS1hc3NpZ25tZW50Lg0K
PiA+DQo+ID4gTm90ZSwgYWxzbywgdGhhdCB0aGUgcGNpX2Fzc2lnbmVkX3ZmcygpIGNoZWNrIGlu
IHRoZSBkcml2ZXJzIHdpbGwNCj4gPiBhbHdheXMgcmV0dXJuIDAgd2hlbiBWRklPIGlzIHVzZWQg
Zm9yIGRldmljZSBhc3NpZ25tZW50LCBzbyBrZWVwaW5nDQo+ID4gdGhlc2UgY2hlY2tzIGluIHRo
ZSBkcml2ZXJzIGRvZXNuJ3QgZG8gd2hhdCB0aGV5IGltcGx5IGVpdGhlci4NCj4gPg0KPiA+IFNv
LCB0YWtpbmcgaW4gdGhlIHBhdGNoIHNvbHZlcyBvbGQsIGt2bS1tYW5hZ2VkLCBkZXZpY2UgYXNz
aWdubWVudCwNCj4gPiBidXQgYSBuZXcgbWV0aG9kIGlzIG5lZWRlZCB3aGVuIFZGSU8gaXMgaW52
b2x2ZWQuDQo+ID4NCj4gPiAtIERvbg0KPiA+DQo+ID4gcHMgLS0gTm90ZToganVzdCBhZGRpbmcg
dGhlIGZsYWcgc2V0dGluZyBpbiB2ZmlvLXBjaSBkb2VzIG5vdCBuZWNlc3NhcmlseQ0KPiA+ICAg
ICAgICBzb2x2ZSB0aGlzIHByb2JsZW0uICBWRklPIGRvZXMgbm90IGtub3cgaWYgYSBkZXZpY2Ug
aXMgYXNzaWduZWQgdG8gYQ0KPiBndWVzdDsNCj4gPiAgICAgICAgaXQgb25seSBrbm93cyBhIGNh
bGxlciBvZiB0aGUgaW9jdGwgcmVxdWVzdGluZyB0aGUgZGV2aWNlIHRvIGJlIGFzc2lnbmVkDQo+
ID4gICAgICAgIHRvIHZmaW8sIGFuZCB0byBiZSBkbWEtbWFwcGVkIGZvciBhIHJlZ2lvbiBvZiBt
ZW1vcnksIGhhcyBiZWVuDQo+IHJlcXVlc3RlZC4NCj4gPiAgICAgICAgU28sIGEgbmV3IFBGPC0+
VkYgbWVjaGFuaXNtIG5lZWRzIHRvIGJlIHB1dCBpbiBwbGFjZSB0bw0KPiA+ICAgICAgICBkZXRl
cm1pbmUgdGhlIGVxdWl2YWxlbnQgaW5mb3JtYXRpb24uDQo+IA0KPiBwcHMgLS0gTm90ZTogdGVz
dGluZyBwY2lfYXNzaXZuZWRfdmZzKCkgaXMgcmFjeSwgbm90aGluZyBwcmV2ZW50cyB0aGUgZmxh
Zw0KPiAJYmVpbmcgYWRkZWQgdG8gYSBkZXZpY2UgYmV0d2VlbiB5b3VyIGNoZWNrIGFuZCByZW1v
dmluZyB0aGUgVkYNCj4gZGV2aWNlLg0KPiAJVGhpcyBpcyBvbmUgb2YgdGhlIHJlYXNvbnMgdGhh
dCB2ZmlvLXBjaSBkb2Vzbid0IHVzZSBpdCBhbmQgdGhhdCB0aGlzDQo+IAlpbnRlcmZhY2Ugc2hv
dWxkIGJlIGRpc2NvdXJhZ2VkIGluIHRoZSBrZXJuZWwuDQoNCkFsZXgvRG9uLCBJIGFncmVlIHdp
dGggdGhlIHBvaW50cyB5b3UndmUgcmFpc2VkLg0KQnV0LCBJJ2QgbGlrZSB0byBrbm93IHdoZXRo
ZXIgeW91IHRoaW5rIHRoaXMgcGF0Y2gtc2V0IHNob3VsZCBiZSBhY2NlcHRlZCBvciBub3QuDQpF
dmVuIHRob3VnaCB0aGlzIHBhdGNoLXNldCBkb2Vzbid0IGZpeCBhbnkgb2YgdGhlIHBlbmRpbmcg
aXNzdWVzIHJhaXNlZCBoZXJlLA0KaXQncyBhIHNtYWxsIHN0ZXAgZm9yd2FyZCBhcyBpdCByZWR1
Y2VzIHRoZSBudW1iZXIgb2YgaW52b2NhdGlvbnMgb2YgcGNpX2Fzc2lnbmVkX3ZmcygpDQpjaGVj
ayB3aGljaCBpcyBhIGdvb2QgdGhpbmcuDQoNCnRoYW5rcywNCi1TYXRoeWENCg==

Re: [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Don Dutile]

On 11/13/2014 02:04 AM, Sathya Perla wrote:
>> -----Original Message-----
>> From: Alex Williamson [mailto:alex.williamson@redhat.com]
>>
>> On Tue, 2014-11-11 at 14:09 -0500, Don Dutile wrote:
>>> On 11/10/2014 06:53 AM, Sathya Perla wrote:
>>>> A user must not be allowed to disable VFs while they are already assigned
>> to
>>>> a guest. This check is being made in each individual driver that
>> implements
>>>> the sriov_configure PCI method.
>>>> This patch-set fixes this code duplication by moving this check from
>>>> drivers to the sriov_nuvfs_store() routine just before invoking
>>>> sriov_configure() when num_vfs is equal to 0.
>>>>
>>>> Vasundhara Volam (4):
>>>>     pci: move pci_assivned_vfs() check while disabling VFs to pci
>>>>       sub-system
>>>>     bnx2x: remove pci_assigned_vfs() check while disabling VFs
>>>>     i40e: remove pci_assigned_vfs() check while disabling VFs
>>>>     qlcnic: remove pci_assigned_vfs() check while disabling VFs
>>>>
>>>>    drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
>>>>    drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
>>>>    .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
>>>>    drivers/pci/pci-sysfs.c                            |    5 +++++
>>>>    4 files changed, 7 insertions(+), 17 deletions(-)
>>>>
>>> I have had a side conversation with Alex Williamson, VFIO author.
>>>
>>> VFIO is the upstream method that device-assignment is managed/handled
>> on kvm now.
>>> It does not set the PCI_DEV_FLAGS_ASSIGNED pci dev-flags, and thus,
>>> this check will not work when VFIO is used.
>>>
>>> This patch set will only work for the former, kvm-managed, device-
>> assignment method,
>>> which is currently being deprecated in qemu as well.
>>>
>>> So, yes, it works for kvm managed device-assignment, but not the
>>> newer, VFIO-based device-assignment.
>>>
>>> Note, also, that the pci_assigned_vfs() check in the drivers will
>>> always return 0 when VFIO is used for device assignment, so keeping
>>> these checks in the drivers doesn't do what they imply either.
>>>
>>> So, taking in the patch solves old, kvm-managed, device assignment,
>>> but a new method is needed when VFIO is involved.
>>>
>>> - Don
>>>
>>> ps -- Note: just adding the flag setting in vfio-pci does not necessarily
>>>         solve this problem.  VFIO does not know if a device is assigned to a
>> guest;
>>>         it only knows a caller of the ioctl requesting the device to be assigned
>>>         to vfio, and to be dma-mapped for a region of memory, has been
>> requested.
>>>         So, a new PF<->VF mechanism needs to be put in place to
>>>         determine the equivalent information.
>>
>> pps -- Note: testing pci_assivned_vfs() is racy, nothing prevents the flag
>> 	being added to a device between your check and removing the VF
>> device.
>> 	This is one of the reasons that vfio-pci doesn't use it and that this
>> 	interface should be discouraged in the kernel.
>
> Alex/Don, I agree with the points you've raised.
> But, I'd like to know whether you think this patch-set should be accepted or not.
> Even though this patch-set doesn't fix any of the pending issues raised here,
> it's a small step forward as it reduces the number of invocations of pci_assigned_vfs()
> check which is a good thing.
>
> thanks,
> -Sathya
>

IMO, it's only a fix for XEN.  Upstream has moved on to VFIO-based device assignment,
and these patches do not fix the issue.  They don't make it any worse, either,
but I don't want someone scanning the patch list thinking it's been fixed either.

So, it's ok (but racy, as it is today) for kvm-managed device-assignment & Xen
pci passthrough, but does zip for VFIO-based assignment.

We need a new api - maybe another/new state in sysfs, so userspace can set it
(like qemu &/or libvirt), as well as (xen) kernel ... and meeting non-racy condition(s).

Re: [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Bjorn Helgaas]

[+cc Jeff]

On Mon, Nov 10, 2014 at 05:23:26PM +0530, Sathya Perla wrote:
> A user must not be allowed to disable VFs while they are already assigned to
> a guest. This check is being made in each individual driver that implements
> the sriov_configure PCI method.
> This patch-set fixes this code duplication by moving this check from
> drivers to the sriov_nuvfs_store() routine just before invoking
> sriov_configure() when num_vfs is equal to 0.
> 
> Vasundhara Volam (4):
>   pci: move pci_assivned_vfs() check while disabling VFs to pci
>     sub-system
>   bnx2x: remove pci_assigned_vfs() check while disabling VFs
>   i40e: remove pci_assigned_vfs() check while disabling VFs
>   qlcnic: remove pci_assigned_vfs() check while disabling VFs
> 
>  drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
>  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
>  .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
>  drivers/pci/pci-sysfs.c                            |    5 +++++
>  4 files changed, 7 insertions(+), 17 deletions(-)

I'm dropping these for the reasons Don & Alex outlined -- they don't fix
the problem for VFIO, so this amounts to shuffling around code that's known
to be broken, which seems more confusing than worthwhile.

Jeff, if I were you I would drop the i40e patch.  I don't think it makes
sense to remove the check from i40e before adding it to the PCI core.

Bjorn

Re: [linux-nics] [PATCH 0/4] move pci_assivned_vfs() check (while disabling VFs) to pci sub-system

[Jeff Kirsher]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Thu, 2014-11-20 at 15:21 -0700, Bjorn Helgaas wrote:
> [+cc Jeff]
> 
> On Mon, Nov 10, 2014 at 05:23:26PM +0530, Sathya Perla wrote:
> > A user must not be allowed to disable VFs while they are already assigned to
> > a guest. This check is being made in each individual driver that implements
> > the sriov_configure PCI method.
> > This patch-set fixes this code duplication by moving this check from
> > drivers to the sriov_nuvfs_store() routine just before invoking
> > sriov_configure() when num_vfs is equal to 0.
> > 
> > Vasundhara Volam (4):
> >   pci: move pci_assivned_vfs() check while disabling VFs to pci
> >     sub-system
> >   bnx2x: remove pci_assigned_vfs() check while disabling VFs
> >   i40e: remove pci_assigned_vfs() check while disabling VFs
> >   qlcnic: remove pci_assigned_vfs() check while disabling VFs
> > 
> >  drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c  |    2 +-
> >  drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c |    7 +------
> >  .../net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c   |   10 ----------
> >  drivers/pci/pci-sysfs.c                            |    5 +++++
> >  4 files changed, 7 insertions(+), 17 deletions(-)
> 
> I'm dropping these for the reasons Don & Alex outlined -- they don't fix
> the problem for VFIO, so this amounts to shuffling around code that's known
> to be broken, which seems more confusing than worthwhile.
> 
> Jeff, if I were you I would drop the i40e patch.  I don't think it makes
> sense to remove the check from i40e before adding it to the PCI core.
> 

Thanks for the heads up, I will drop the i40e patch as well.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]