From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: From: "Raslan, KarimAllah" To: "helgaas@kernel.org" CC: "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "bhelgaas@google.com" Subject: Re: [PATCH v2] pci: Store more data about VFs into the SRIOV struct Date: Thu, 1 Mar 2018 21:32:02 +0000 Message-ID: <1519939922.15908.51.camel@amazon.de> References: <1519910764-12789-1-git-send-email-karahmed@amazon.de> <20180301193405.GL13722@bhelgaas-glaptop.roam.corp.google.com> In-Reply-To: <20180301193405.GL13722@bhelgaas-glaptop.roam.corp.google.com> Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 List-ID: T24gVGh1LCAyMDE4LTAzLTAxIGF0IDEzOjM0IC0wNjAwLCBCam9ybiBIZWxnYWFzIHdyb3RlOg0K PiBzfHBjaTogU3RvcmV8UENJL0lPVjogU3RvcmV8DQo+IA0KPiAocnVuICJnaXQgbG9nIC0tb25l bGluZSBkcml2ZXJzL3BjaS9wcm9iZS5jIiB0byBzZWUgd2h5KQ0KPiANCj4gT24gVGh1LCBNYXIg MDEsIDIwMTggYXQgMDI6MjY6MDRQTSArMDEwMCwgS2FyaW1BbGxhaCBBaG1lZCB3cm90ZToNCj4g PiANCj4gPiAuLi4gdG8gYXZvaWQgcmVhZGluZyB0aGVtIGZyb20gdGhlIGNvbmZpZyBzcGFjZSBv ZiBhbGwgdGhlIFBDSSBWRnMuIFRoaXMgaXMNCj4gPiBzcGVjaWFsbHkgYSB1c2VmdWwgb3B0aW1p emF0aW9uIHdoZW4gYnJpbmdpbmcgdXAgdGhvdXNhbmRzIG9mIFZGcy4NCj4gDQo+IFBsZWFzZSBt YWtlIHRoZSBjaGFuZ2Vsb2cgY29tcGxldGUgaW4gaXRzZWxmLCBzbyBpdCBkb2Vzbid0IGhhdmUg dG8gYmUNCj4gcmVhZCBpbiBjb25qdW5jdGlvbiB3aXRoIHRoZSBzdWJqZWN0LiAgSXQncyBPSyBp ZiB5b3UgaGF2ZSB0byByZXBlYXQNCj4gdGhlIHN1YmplY3QgaW4gdGhlIGNoYW5nZWxvZy4NCg0K YWNrLg0KDQo+IA0KPiA+IA0KPiA+IENjOiBCam9ybiBIZWxnYWFzIDxiaGVsZ2Fhc0Bnb29nbGUu Y29tPg0KPiA+IENjOiBsaW51eC1wY2lAdmdlci5rZXJuZWwub3JnDQo+ID4gQ2M6IGxpbnV4LWtl cm5lbEB2Z2VyLmtlcm5lbC5vcmcNCj4gPiBTaWduZWQtb2ZmLWJ5OiBLYXJpbUFsbGFoIEFobWVk IDxrYXJhaG1lZEBhbWF6b24uZGU+DQo+ID4gLS0tDQo+ID4gdjEgLT4gdjI6DQo+ID4gKiBSZWJh c2Ugb24gbGF0ZXN0ICsgcmVtb3ZlIGRlcGVuZGVuY3kgb24gYSBub24tdXBzdHJlYW0gcGF0Y2gu DQo+ID4gDQo+ID4gIGRyaXZlcnMvcGNpL2lvdi5jICAgfCAxNiArKysrKysrKysrKysrKysrDQo+ ID4gIGRyaXZlcnMvcGNpL3BjaS5oICAgfCAgNSArKysrKw0KPiA+ICBkcml2ZXJzL3BjaS9wcm9i ZS5jIHwgNDIgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0tLS0tDQo+ID4g IDMgZmlsZXMgY2hhbmdlZCwgNTMgaW5zZXJ0aW9ucygrKSwgMTAgZGVsZXRpb25zKC0pDQo+ID4g DQo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvcGNpL2lvdi5jIGIvZHJpdmVycy9wY2kvaW92LmMN Cj4gPiBpbmRleCA2Nzc5MjRhLi5lMWQyZTNmIDEwMDY0NA0KPiA+IC0tLSBhL2RyaXZlcnMvcGNp L2lvdi5jDQo+ID4gKysrIGIvZHJpdmVycy9wY2kvaW92LmMNCj4gPiBAQCAtMTE0LDYgKzExNCwx OSBAQCByZXNvdXJjZV9zaXplX3QgcGNpX2lvdl9yZXNvdXJjZV9zaXplKHN0cnVjdCBwY2lfZGV2 ICpkZXYsIGludCByZXNubykNCj4gPiAgCXJldHVybiBkZXYtPnNyaW92LT5iYXJzeltyZXNubyAt IFBDSV9JT1ZfUkVTT1VSQ0VTXTsNCj4gPiAgfQ0KPiA+ICANCj4gPiArc3RhdGljIHZvaWQgcGNp X3JlYWRfdmZfY29uZmlnX2NvbW1vbihzdHJ1Y3QgcGNpX2J1cyAqYnVzLCBzdHJ1Y3QgcGNpX2Rl diAqZGV2KQ0KPiA+ICt7DQo+ID4gKwlpbnQgZGV2Zm4gPSBwY2lfaW92X3ZpcnRmbl9kZXZmbihk ZXYsIDApOw0KPiA+ICsNCj4gPiArCXBjaV9idXNfcmVhZF9jb25maWdfZHdvcmQoYnVzLCBkZXZm biwgUENJX0NMQVNTX1JFVklTSU9OLA0KPiA+ICsJCQkJICAmZGV2LT5zcmlvdi0+Y2xhc3MpOw0K PiA+ICsJcGNpX2J1c19yZWFkX2NvbmZpZ193b3JkKGJ1cywgZGV2Zm4sIFBDSV9TVUJTWVNURU1f SUQsDQo+ID4gKwkJCQkgJmRldi0+c3Jpb3YtPnN1YnN5c3RlbV9kZXZpY2UpOw0KPiA+ICsJcGNp X2J1c19yZWFkX2NvbmZpZ193b3JkKGJ1cywgZGV2Zm4sIFBDSV9TVUJTWVNURU1fVkVORE9SX0lE LA0KPiA+ICsJCQkJICZkZXYtPnNyaW92LT5zdWJzeXN0ZW1fdmVuZG9yKTsNCj4gPiArCXBjaV9i dXNfcmVhZF9jb25maWdfYnl0ZShidXMsIGRldmZuLCBQQ0lfSEVBREVSX1RZUEUsICZkZXYtPnNy aW92LT5oZHJfdHlwZSk7DQo+IA0KPiBDYW4ndCB5b3UgZG8gdGhpcyBhIGxpdHRsZSBsYXRlciwg ZS5nLiwgYWZ0ZXIgcGNpX2lvdl9hZGRfdmlydGZuKCkNCj4gY2FsbHMgcGNpX3NldHVwX2Rldmlj ZSgpLCBhbmQgdGhlbiB1c2UgdGhlIHN0YW5kYXJkDQo+IHBjaV9yZWFkX2NvbmZpZ18qKCkgaW50 ZXJmYWNlcyBpbnN0ZWFkIG9mIHRoZSBzcGVjaWFsDQo+IHBjaV9idXNfcmVhZF9jb25maWcqKCkg b25lcz8NCg0KYWNrLg0KDQpJIG1vdmVkIGl0IGFmdGVyICJwY2lfaW92X3ZpcnRmbl9kZXZmbiIu DQoNCj4gDQo+ID4gDQo+ID4gK30NCj4gPiArDQo+ID4gIGludCBwY2lfaW92X2FkZF92aXJ0Zm4o c3RydWN0IHBjaV9kZXYgKmRldiwgaW50IGlkKQ0KPiA+ICB7DQo+ID4gIAlpbnQgaTsNCj4gPiBA QCAtMTMzLDYgKzE0Niw5IEBAIGludCBwY2lfaW92X2FkZF92aXJ0Zm4oc3RydWN0IHBjaV9kZXYg KmRldiwgaW50IGlkKQ0KPiA+ICAJaWYgKCF2aXJ0Zm4pDQo+ID4gIAkJZ290byBmYWlsZWQwOw0K PiA+ICANCj4gPiArCWlmIChpZCA9PSAwKQ0KPiA+ICsJCXBjaV9yZWFkX3ZmX2NvbmZpZ19jb21t b24oYnVzLCBkZXYpOw0KPiA+ICsNCj4gPiAgCXZpcnRmbi0+ZGV2Zm4gPSBwY2lfaW92X3ZpcnRm bl9kZXZmbihkZXYsIGlkKTsNCj4gPiAgCXZpcnRmbi0+dmVuZG9yID0gZGV2LT52ZW5kb3I7DQo+ ID4gIAl2aXJ0Zm4tPmRldmljZSA9IGlvdi0+dmZfZGV2aWNlOw0KPiA+IGRpZmYgLS1naXQgYS9k cml2ZXJzL3BjaS9wY2kuaCBiL2RyaXZlcnMvcGNpL3BjaS5oDQo+ID4gaW5kZXggZmNkODE5MS4u MzQ2ZGFhNSAxMDA2NDQNCj4gPiAtLS0gYS9kcml2ZXJzL3BjaS9wY2kuaA0KPiA+ICsrKyBiL2Ry aXZlcnMvcGNpL3BjaS5oDQo+ID4gQEAgLTI3MSw2ICsyNzEsMTEgQEAgc3RydWN0IHBjaV9zcmlv diB7DQo+ID4gIAl1MTYJCWRyaXZlcl9tYXhfVkZzOwkvKiBNYXggbnVtIFZGcyBkcml2ZXIgc3Vw cG9ydHMgKi8NCj4gPiAgCXN0cnVjdCBwY2lfZGV2CSpkZXY7CQkvKiBMb3dlc3QgbnVtYmVyZWQg UEYgKi8NCj4gPiAgCXN0cnVjdCBwY2lfZGV2CSpzZWxmOwkJLyogVGhpcyBQRiAqLw0KPiA+ICsJ dTggaGRyX3R5cGU7CQkvKiBWRiBoZWFkZXIgdHlwZSAqLw0KPiA+ICsJdTMyIGNsYXNzOwkJLyog VkYgZGV2aWNlICovDQo+ID4gKwl1MTYgZGV2aWNlOwkJLyogVkYgZGV2aWNlICovDQo+ID4gKwl1 MTYgc3Vic3lzdGVtX3ZlbmRvcjsJLyogVkYgc3Vic3lzdGVtIHZlbmRvciAqLw0KPiA+ICsJdTE2 IHN1YnN5c3RlbV9kZXZpY2U7CS8qIFZGIHN1YnN5c3RlbSBkZXZpY2UgKi8NCj4gDQo+IFBsZWFz ZSBtYWtlIHRoZSB3aGl0ZXNwYWNlIGhlcmUgbWF0Y2ggdGhlIGV4aXN0aW5nIGNvZGUsIGkuZS4s DQo+IGxpbmUgdXAgdGhlIHN0cnVjdHVyZSBlbGVtZW50IG5hbWVzIGFuZCBjb21tZW50cy4NCg0K YWNrIQ0KDQo+IA0KPiA+IA0KPiA+ICAJcmVzb3VyY2Vfc2l6ZV90CWJhcnN6W1BDSV9TUklPVl9O VU1fQkFSU107CS8qIFZGIEJBUiBzaXplICovDQo+ID4gIAlib29sCQlkcml2ZXJzX2F1dG9wcm9i ZTsgLyogQXV0byBwcm9iaW5nIG9mIFZGcyBieSBkcml2ZXIgKi8NCj4gPiAgfTsNCj4gPiBkaWZm IC0tZ2l0IGEvZHJpdmVycy9wY2kvcHJvYmUuYyBiL2RyaXZlcnMvcGNpL3Byb2JlLmMNCj4gPiBp bmRleCBlZjUzNzc0Li5hZWFhMTBhIDEwMDY0NA0KPiA+IC0tLSBhL2RyaXZlcnMvcGNpL3Byb2Jl LmMNCj4gPiArKysgYi9kcml2ZXJzL3BjaS9wcm9iZS5jDQo+ID4gQEAgLTE4MCw2ICsxODAsNyBA QCBzdGF0aWMgaW5saW5lIHVuc2lnbmVkIGxvbmcgZGVjb2RlX2JhcihzdHJ1Y3QgcGNpX2RldiAq ZGV2LCB1MzIgYmFyKQ0KPiA+ICBpbnQgX19wY2lfcmVhZF9iYXNlKHN0cnVjdCBwY2lfZGV2ICpk ZXYsIGVudW0gcGNpX2Jhcl90eXBlIHR5cGUsDQo+ID4gIAkJICAgIHN0cnVjdCByZXNvdXJjZSAq cmVzLCB1bnNpZ25lZCBpbnQgcG9zKQ0KPiA+ICB7DQo+ID4gKwlpbnQgYmFyID0gcmVzIC0gZGV2 LT5yZXNvdXJjZTsNCj4gPiAgCXUzMiBsID0gMCwgc3ogPSAwLCBtYXNrOw0KPiA+ICAJdTY0IGw2 NCwgc3o2NCwgbWFzazY0Ow0KPiA+ICAJdTE2IG9yaWdfY21kOw0KPiA+IEBAIC0xOTksOSArMjAw LDEzIEBAIGludCBfX3BjaV9yZWFkX2Jhc2Uoc3RydWN0IHBjaV9kZXYgKmRldiwgZW51bSBwY2lf YmFyX3R5cGUgdHlwZSwNCj4gPiAgCXJlcy0+bmFtZSA9IHBjaV9uYW1lKGRldik7DQo+ID4gIA0K PiA+ICAJcGNpX3JlYWRfY29uZmlnX2R3b3JkKGRldiwgcG9zLCAmbCk7DQo+ID4gLQlwY2lfd3Jp dGVfY29uZmlnX2R3b3JkKGRldiwgcG9zLCBsIHwgbWFzayk7DQo+ID4gLQlwY2lfcmVhZF9jb25m aWdfZHdvcmQoZGV2LCBwb3MsICZzeik7DQo+ID4gLQlwY2lfd3JpdGVfY29uZmlnX2R3b3JkKGRl diwgcG9zLCBsKTsNCj4gPiArCWlmIChkZXYtPmlzX3ZpcnRmbikgew0KPiA+ICsJCXN6ID0gZGV2 LT5waHlzZm4tPnNyaW92LT5iYXJzeltiYXJdICYgMHhmZmZmZmZmZjsNCj4gPiArCX0gZWxzZSB7 DQo+ID4gKwkJcGNpX3dyaXRlX2NvbmZpZ19kd29yZChkZXYsIHBvcywgbCB8IG1hc2spOw0KPiA+ ICsJCXBjaV9yZWFkX2NvbmZpZ19kd29yZChkZXYsIHBvcywgJnN6KTsNCj4gPiArCQlwY2lfd3Jp dGVfY29uZmlnX2R3b3JkKGRldiwgcG9zLCBsKTsNCj4gPiArCX0NCj4gDQo+IFRoaXMgcGFydCBp cyBub3QgbGlrZSB0aGUgb3RoZXJzLCBpLmUuLCB0aGUgb3RoZXJzIGFyZSBjYWNoaW5nIGluZm8N Cj4gZnJvbSBWRiAwIGluIG5ld2x5LWFkZGVkIGVsZW1lbnRzIG9mIHN0cnVjdCBwY2lfc3Jpb3Yu ICBUaGlzIGFsc28gdXNlcw0KPiBpbmZvcm1hdGlvbiBmcm9tIHN0cnVjdCBwY2lfc3Jpb3YsIGJ1 dCBpdCdzIHF1YWxpdGF0aXZlbHkgZGlmZmVyZW50LA0KPiBzbyBpdCBzaG91bGQgYmUgaW4gYSBz ZXBhcmF0ZSBwYXRjaC4NCg0KYWNrLiBNb3ZlZCB0byBhIHNlcGVyYXRlIHBhdGNoLg0KDQo+IA0K PiA+IA0KPiA+ICAJLyoNCj4gPiAgCSAqIEFsbCBiaXRzIHNldCBpbiBzeiBtZWFucyB0aGUgZGV2 aWNlIGlzbid0IHdvcmtpbmcgcHJvcGVybHkuDQo+ID4gQEAgLTI0MSw5ICsyNDYsMTQgQEAgaW50 IF9fcGNpX3JlYWRfYmFzZShzdHJ1Y3QgcGNpX2RldiAqZGV2LCBlbnVtIHBjaV9iYXJfdHlwZSB0 eXBlLA0KPiA+ICANCj4gPiAgCWlmIChyZXMtPmZsYWdzICYgSU9SRVNPVVJDRV9NRU1fNjQpIHsN Cj4gPiAgCQlwY2lfcmVhZF9jb25maWdfZHdvcmQoZGV2LCBwb3MgKyA0LCAmbCk7DQo+ID4gLQkJ cGNpX3dyaXRlX2NvbmZpZ19kd29yZChkZXYsIHBvcyArIDQsIH4wKTsNCj4gPiAtCQlwY2lfcmVh ZF9jb25maWdfZHdvcmQoZGV2LCBwb3MgKyA0LCAmc3opOw0KPiA+IC0JCXBjaV93cml0ZV9jb25m aWdfZHdvcmQoZGV2LCBwb3MgKyA0LCBsKTsNCj4gPiArDQo+ID4gKwkJaWYgKGRldi0+aXNfdmly dGZuKSB7DQo+ID4gKwkJCXN6ID0gKGRldi0+cGh5c2ZuLT5zcmlvdi0+YmFyc3pbYmFyXSA+PiAz MikgJiAweGZmZmZmZmZmOw0KPiA+ICsJCX0gZWxzZSB7DQo+ID4gKwkJCXBjaV93cml0ZV9jb25m aWdfZHdvcmQoZGV2LCBwb3MgKyA0LCB+MCk7DQo+ID4gKwkJCXBjaV9yZWFkX2NvbmZpZ19kd29y ZChkZXYsIHBvcyArIDQsICZzeik7DQo+ID4gKwkJCXBjaV93cml0ZV9jb25maWdfZHdvcmQoZGV2 LCBwb3MgKyA0LCBsKTsNCj4gPiArCQl9DQo+ID4gIA0KPiA+ICAJCWw2NCB8PSAoKHU2NClsIDw8 IDMyKTsNCj4gPiAgCQlzejY0IHw9ICgodTY0KXN6IDw8IDMyKTsNCj4gPiBAQCAtMzMyLDYgKzM0 Miw4IEBAIHN0YXRpYyB2b2lkIHBjaV9yZWFkX2Jhc2VzKHN0cnVjdCBwY2lfZGV2ICpkZXYsIHVu c2lnbmVkIGludCBob3dtYW55LCBpbnQgcm9tKQ0KPiA+ICAJZm9yIChwb3MgPSAwOyBwb3MgPCBo b3dtYW55OyBwb3MrKykgew0KPiA+ICAJCXN0cnVjdCByZXNvdXJjZSAqcmVzID0gJmRldi0+cmVz b3VyY2VbcG9zXTsNCj4gPiAgCQlyZWcgPSBQQ0lfQkFTRV9BRERSRVNTXzAgKyAocG9zIDw8IDIp Ow0KPiA+ICsJCWlmIChkZXYtPmlzX3ZpcnRmbiAmJiBkZXYtPnBoeXNmbi0+c3Jpb3YtPmJhcnN6 W3Bvc10gPT0gMCkNCj4gPiArCQkJY29udGludWU7DQo+ID4gIAkJcG9zICs9IF9fcGNpX3JlYWRf YmFzZShkZXYsIHBjaV9iYXJfdW5rbm93biwgcmVzLCByZWcpOw0KPiA+ICAJfQ0KPiA+ICANCj4g PiBAQCAtMTQ1NCw3ICsxNDY2LDkgQEAgaW50IHBjaV9zZXR1cF9kZXZpY2Uoc3RydWN0IHBjaV9k ZXYgKmRldikNCj4gPiAgCXN0cnVjdCBwY2lfYnVzX3JlZ2lvbiByZWdpb247DQo+ID4gIAlzdHJ1 Y3QgcmVzb3VyY2UgKnJlczsNCj4gPiAgDQo+ID4gLQlpZiAocGNpX3JlYWRfY29uZmlnX2J5dGUo ZGV2LCBQQ0lfSEVBREVSX1RZUEUsICZoZHJfdHlwZSkpDQo+ID4gKwlpZiAoZGV2LT5pc192aXJ0 Zm4pDQo+ID4gKwkJaGRyX3R5cGUgPSBkZXYtPnBoeXNmbi0+c3Jpb3YtPmhkcl90eXBlOw0KPiA+ ICsJZWxzZSBpZiAocGNpX3JlYWRfY29uZmlnX2J5dGUoZGV2LCBQQ0lfSEVBREVSX1RZUEUsICZo ZHJfdHlwZSkpDQo+ID4gIAkJcmV0dXJuIC1FSU87DQo+ID4gIA0KPiA+ICAJZGV2LT5zeXNkYXRh ID0gZGV2LT5idXMtPnN5c2RhdGE7DQo+ID4gQEAgLTE0NzcsNyArMTQ5MSwxMCBAQCBpbnQgcGNp X3NldHVwX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPiA+ICAJCSAgICAgZGV2LT5idXMt Pm51bWJlciwgUENJX1NMT1QoZGV2LT5kZXZmbiksDQo+ID4gIAkJICAgICBQQ0lfRlVOQyhkZXYt PmRldmZuKSk7DQo+ID4gIA0KPiA+IC0JcGNpX3JlYWRfY29uZmlnX2R3b3JkKGRldiwgUENJX0NM QVNTX1JFVklTSU9OLCAmY2xhc3MpOw0KPiA+ICsJaWYgKGRldi0+aXNfdmlydGZuKQ0KPiA+ICsJ CWNsYXNzID0gZGV2LT5waHlzZm4tPnNyaW92LT5jbGFzczsNCj4gPiArCWVsc2UNCj4gPiArCQlw Y2lfcmVhZF9jb25maWdfZHdvcmQoZGV2LCBQQ0lfQ0xBU1NfUkVWSVNJT04sICZjbGFzcyk7DQo+ ID4gIAlkZXYtPnJldmlzaW9uID0gY2xhc3MgJiAweGZmOw0KPiA+ICAJZGV2LT5jbGFzcyA9IGNs YXNzID4+IDg7CQkgICAgLyogdXBwZXIgMyBieXRlcyAqLw0KPiA+ICANCj4gPiBAQCAtMTUxNyw4 ICsxNTM0LDEzIEBAIGludCBwY2lfc2V0dXBfZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+ ID4gIAkJCWdvdG8gYmFkOw0KPiA+ICAJCXBjaV9yZWFkX2lycShkZXYpOw0KPiA+ICAJCXBjaV9y ZWFkX2Jhc2VzKGRldiwgNiwgUENJX1JPTV9BRERSRVNTKTsNCj4gPiAtCQlwY2lfcmVhZF9jb25m aWdfd29yZChkZXYsIFBDSV9TVUJTWVNURU1fVkVORE9SX0lELCAmZGV2LT5zdWJzeXN0ZW1fdmVu ZG9yKTsNCj4gPiAtCQlwY2lfcmVhZF9jb25maWdfd29yZChkZXYsIFBDSV9TVUJTWVNURU1fSUQs ICZkZXYtPnN1YnN5c3RlbV9kZXZpY2UpOw0KPiA+ICsJCWlmIChkZXYtPmlzX3ZpcnRmbikgew0K PiA+ICsJCQlkZXYtPnN1YnN5c3RlbV92ZW5kb3IgPSBkZXYtPnBoeXNmbi0+c3Jpb3YtPnN1YnN5 c3RlbV92ZW5kb3I7DQo+ID4gKwkJCWRldi0+c3Vic3lzdGVtX2RldmljZSA9IGRldi0+cGh5c2Zu LT5zcmlvdi0+c3Vic3lzdGVtX2RldmljZTsNCj4gDQo+IFBDSWUgcjQuMCwgc2VjIDkuMy40LjEu MTMgcmVxdWlyZXMgdGhhdCBTdWJzeXN0ZW0gVmVuZG9yIElEIGJlIHRoZQ0KPiBzYW1lIGZvciB0 aGUgUEYgYW5kIGFsbCBWRnMsIGJ1dCBzZWMgOS4zLjQuMS4xNCBzYXlzIHRoZSBQRiBhbmQgVkYg bWF5DQo+IGhhdmUgZGlmZmVyZW50IFN1YnN5c3RlbSBJRHMuICBJIGtub3cgeW91J3JlIGNhY2hp bmcgdGhlIFN1YnN5c3RlbSBJRA0KPiBmcm9tIFZGIDAsIG5vdCB0aGUgUEYsIGJ1dCBJIGRvbid0 IHNlZSBhbnl0aGluZyB0aGF0IHJlcXVpcmVzIGFsbCB0aGUNCj4gVkZzIHRvIGhhdmUgdGhlIHNh bWUgU3Vic3lzdGVtIElELg0KPiANCj4gSSB0aGluayB0aGUgc2FtZSBpcyB0ZWNobmljYWxseSB0 cnVlIGZvciB0aGUgUmV2aXNpb24gSUQuICBJdCBtaWdodCBiZQ0KPiByZWFzb25hYmxlIHRvIGFz c3VtZSBhbGwgdGhlIFZGcyBoYXZlIHRoZSBzYW1lIHZhbHVlcywgYnV0IG1heWJlIHdvcnRoDQo+ IGEgY29tbWVudC4NCg0KSSBhZGRlZCBhIGNvbW1lbnQgYWJvdXQgdGhhdCBmb3IgdGhlIDMgZmll bGRzLg0KDQo+IA0KPiA+IA0KPiA+ICsJCX0gZWxzZSB7DQo+ID4gKwkJCXBjaV9yZWFkX2NvbmZp Z193b3JkKGRldiwgUENJX1NVQlNZU1RFTV9WRU5ET1JfSUQsICZkZXYtPnN1YnN5c3RlbV92ZW5k b3IpOw0KPiA+ICsJCQlwY2lfcmVhZF9jb25maWdfd29yZChkZXYsIFBDSV9TVUJTWVNURU1fSUQs ICZkZXYtPnN1YnN5c3RlbV9kZXZpY2UpOw0KPiA+ICsJCX0NCj4gPiAgDQo+ID4gIAkJLyoNCj4g PiAgCQkgKiBEbyB0aGUgdWdseSBsZWdhY3kgbW9kZSBzdHVmZiBoZXJlIHJhdGhlciB0aGFuIGJy b2tlbiBjaGlwDQo+ID4gLS0gDQo+ID4gMi43LjQNCj4gPiANCj4gDQpBbWF6b24gRGV2ZWxvcG1l bnQgQ2VudGVyIEdlcm1hbnkgR21iSApCZXJsaW4gLSBEcmVzZGVuIC0gQWFjaGVuCm1haW4gb2Zm aWNlOiBLcmF1c2Vuc3RyLiAzOCwgMTAxMTcgQmVybGluCkdlc2NoYWVmdHNmdWVocmVyOiBEci4g UmFsZiBIZXJicmljaCwgQ2hyaXN0aWFuIFNjaGxhZWdlcgpVc3QtSUQ6IERFMjg5MjM3ODc5CkVp bmdldHJhZ2VuIGFtIEFtdHNnZXJpY2h0IENoYXJsb3R0ZW5idXJnIEhSQiAxNDkxNzMgQgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162106AbeCAVcS (ORCPT ); Thu, 1 Mar 2018 16:32:18 -0500 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:40396 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161792AbeCAVcM (ORCPT ); Thu, 1 Mar 2018 16:32:12 -0500 X-IronPort-AV: E=Sophos;i="5.47,409,1515456000"; d="scan'208";a="668874718" From: "Raslan, KarimAllah" To: "helgaas@kernel.org" CC: "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "bhelgaas@google.com" Subject: Re: [PATCH v2] pci: Store more data about VFs into the SRIOV struct Thread-Topic: [PATCH v2] pci: Store more data about VFs into the SRIOV struct Thread-Index: AQHTsWDs7fDQMe1Am0i79I3K4P45AaO7xcKAgAAg9QA= Date: Thu, 1 Mar 2018 21:32:02 +0000 Message-ID: <1519939922.15908.51.camel@amazon.de> References: <1519910764-12789-1-git-send-email-karahmed@amazon.de> <20180301193405.GL13722@bhelgaas-glaptop.roam.corp.google.com> In-Reply-To: <20180301193405.GL13722@bhelgaas-glaptop.roam.corp.google.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.43.164.111] Content-Type: text/plain; charset="utf-8" Content-ID: MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id w21LWMMP024854 On Thu, 2018-03-01 at 13:34 -0600, Bjorn Helgaas wrote: > s|pci: Store|PCI/IOV: Store| > > (run "git log --oneline drivers/pci/probe.c" to see why) > > On Thu, Mar 01, 2018 at 02:26:04PM +0100, KarimAllah Ahmed wrote: > > > > ... to avoid reading them from the config space of all the PCI VFs. This is > > specially a useful optimization when bringing up thousands of VFs. > > Please make the changelog complete in itself, so it doesn't have to be > read in conjunction with the subject. It's OK if you have to repeat > the subject in the changelog. ack. > > > > > Cc: Bjorn Helgaas > > Cc: linux-pci@vger.kernel.org > > Cc: linux-kernel@vger.kernel.org > > Signed-off-by: KarimAllah Ahmed > > --- > > v1 -> v2: > > * Rebase on latest + remove dependency on a non-upstream patch. > > > > drivers/pci/iov.c | 16 ++++++++++++++++ > > drivers/pci/pci.h | 5 +++++ > > drivers/pci/probe.c | 42 ++++++++++++++++++++++++++++++++---------- > > 3 files changed, 53 insertions(+), 10 deletions(-) > > > > diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c > > index 677924a..e1d2e3f 100644 > > --- a/drivers/pci/iov.c > > +++ b/drivers/pci/iov.c > > @@ -114,6 +114,19 @@ resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno) > > return dev->sriov->barsz[resno - PCI_IOV_RESOURCES]; > > } > > > > +static void pci_read_vf_config_common(struct pci_bus *bus, struct pci_dev *dev) > > +{ > > + int devfn = pci_iov_virtfn_devfn(dev, 0); > > + > > + pci_bus_read_config_dword(bus, devfn, PCI_CLASS_REVISION, > > + &dev->sriov->class); > > + pci_bus_read_config_word(bus, devfn, PCI_SUBSYSTEM_ID, > > + &dev->sriov->subsystem_device); > > + pci_bus_read_config_word(bus, devfn, PCI_SUBSYSTEM_VENDOR_ID, > > + &dev->sriov->subsystem_vendor); > > + pci_bus_read_config_byte(bus, devfn, PCI_HEADER_TYPE, &dev->sriov->hdr_type); > > Can't you do this a little later, e.g., after pci_iov_add_virtfn() > calls pci_setup_device(), and then use the standard > pci_read_config_*() interfaces instead of the special > pci_bus_read_config*() ones? ack. I moved it after "pci_iov_virtfn_devfn". > > > > > +} > > + > > int pci_iov_add_virtfn(struct pci_dev *dev, int id) > > { > > int i; > > @@ -133,6 +146,9 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id) > > if (!virtfn) > > goto failed0; > > > > + if (id == 0) > > + pci_read_vf_config_common(bus, dev); > > + > > virtfn->devfn = pci_iov_virtfn_devfn(dev, id); > > virtfn->vendor = dev->vendor; > > virtfn->device = iov->vf_device; > > diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h > > index fcd8191..346daa5 100644 > > --- a/drivers/pci/pci.h > > +++ b/drivers/pci/pci.h > > @@ -271,6 +271,11 @@ struct pci_sriov { > > u16 driver_max_VFs; /* Max num VFs driver supports */ > > struct pci_dev *dev; /* Lowest numbered PF */ > > struct pci_dev *self; /* This PF */ > > + u8 hdr_type; /* VF header type */ > > + u32 class; /* VF device */ > > + u16 device; /* VF device */ > > + u16 subsystem_vendor; /* VF subsystem vendor */ > > + u16 subsystem_device; /* VF subsystem device */ > > Please make the whitespace here match the existing code, i.e., > line up the structure element names and comments. ack! > > > > > resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */ > > bool drivers_autoprobe; /* Auto probing of VFs by driver */ > > }; > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > > index ef53774..aeaa10a 100644 > > --- a/drivers/pci/probe.c > > +++ b/drivers/pci/probe.c > > @@ -180,6 +180,7 @@ static inline unsigned long decode_bar(struct pci_dev *dev, u32 bar) > > int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type, > > struct resource *res, unsigned int pos) > > { > > + int bar = res - dev->resource; > > u32 l = 0, sz = 0, mask; > > u64 l64, sz64, mask64; > > u16 orig_cmd; > > @@ -199,9 +200,13 @@ int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type, > > res->name = pci_name(dev); > > > > pci_read_config_dword(dev, pos, &l); > > - pci_write_config_dword(dev, pos, l | mask); > > - pci_read_config_dword(dev, pos, &sz); > > - pci_write_config_dword(dev, pos, l); > > + if (dev->is_virtfn) { > > + sz = dev->physfn->sriov->barsz[bar] & 0xffffffff; > > + } else { > > + pci_write_config_dword(dev, pos, l | mask); > > + pci_read_config_dword(dev, pos, &sz); > > + pci_write_config_dword(dev, pos, l); > > + } > > This part is not like the others, i.e., the others are caching info > from VF 0 in newly-added elements of struct pci_sriov. This also uses > information from struct pci_sriov, but it's qualitatively different, > so it should be in a separate patch. ack. Moved to a seperate patch. > > > > > /* > > * All bits set in sz means the device isn't working properly. > > @@ -241,9 +246,14 @@ int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type, > > > > if (res->flags & IORESOURCE_MEM_64) { > > pci_read_config_dword(dev, pos + 4, &l); > > - pci_write_config_dword(dev, pos + 4, ~0); > > - pci_read_config_dword(dev, pos + 4, &sz); > > - pci_write_config_dword(dev, pos + 4, l); > > + > > + if (dev->is_virtfn) { > > + sz = (dev->physfn->sriov->barsz[bar] >> 32) & 0xffffffff; > > + } else { > > + pci_write_config_dword(dev, pos + 4, ~0); > > + pci_read_config_dword(dev, pos + 4, &sz); > > + pci_write_config_dword(dev, pos + 4, l); > > + } > > > > l64 |= ((u64)l << 32); > > sz64 |= ((u64)sz << 32); > > @@ -332,6 +342,8 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom) > > for (pos = 0; pos < howmany; pos++) { > > struct resource *res = &dev->resource[pos]; > > reg = PCI_BASE_ADDRESS_0 + (pos << 2); > > + if (dev->is_virtfn && dev->physfn->sriov->barsz[pos] == 0) > > + continue; > > pos += __pci_read_base(dev, pci_bar_unknown, res, reg); > > } > > > > @@ -1454,7 +1466,9 @@ int pci_setup_device(struct pci_dev *dev) > > struct pci_bus_region region; > > struct resource *res; > > > > - if (pci_read_config_byte(dev, PCI_HEADER_TYPE, &hdr_type)) > > + if (dev->is_virtfn) > > + hdr_type = dev->physfn->sriov->hdr_type; > > + else if (pci_read_config_byte(dev, PCI_HEADER_TYPE, &hdr_type)) > > return -EIO; > > > > dev->sysdata = dev->bus->sysdata; > > @@ -1477,7 +1491,10 @@ int pci_setup_device(struct pci_dev *dev) > > dev->bus->number, PCI_SLOT(dev->devfn), > > PCI_FUNC(dev->devfn)); > > > > - pci_read_config_dword(dev, PCI_CLASS_REVISION, &class); > > + if (dev->is_virtfn) > > + class = dev->physfn->sriov->class; > > + else > > + pci_read_config_dword(dev, PCI_CLASS_REVISION, &class); > > dev->revision = class & 0xff; > > dev->class = class >> 8; /* upper 3 bytes */ > > > > @@ -1517,8 +1534,13 @@ int pci_setup_device(struct pci_dev *dev) > > goto bad; > > pci_read_irq(dev); > > pci_read_bases(dev, 6, PCI_ROM_ADDRESS); > > - pci_read_config_word(dev, PCI_SUBSYSTEM_VENDOR_ID, &dev->subsystem_vendor); > > - pci_read_config_word(dev, PCI_SUBSYSTEM_ID, &dev->subsystem_device); > > + if (dev->is_virtfn) { > > + dev->subsystem_vendor = dev->physfn->sriov->subsystem_vendor; > > + dev->subsystem_device = dev->physfn->sriov->subsystem_device; > > PCIe r4.0, sec 9.3.4.1.13 requires that Subsystem Vendor ID be the > same for the PF and all VFs, but sec 9.3.4.1.14 says the PF and VF may > have different Subsystem IDs. I know you're caching the Subsystem ID > from VF 0, not the PF, but I don't see anything that requires all the > VFs to have the same Subsystem ID. > > I think the same is technically true for the Revision ID. It might be > reasonable to assume all the VFs have the same values, but maybe worth > a comment. I added a comment about that for the 3 fields. > > > > > + } else { > > + pci_read_config_word(dev, PCI_SUBSYSTEM_VENDOR_ID, &dev->subsystem_vendor); > > + pci_read_config_word(dev, PCI_SUBSYSTEM_ID, &dev->subsystem_device); > > + } > > > > /* > > * Do the ugly legacy mode stuff here rather than broken chip > > -- > > 2.7.4 > > > Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B