From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:53305 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726186AbgA2MJ5 (ORCPT ); Wed, 29 Jan 2020 07:09:57 -0500 Date: Wed, 29 Jan 2020 13:09:45 +0100 From: Jesper Dangaard Brouer Subject: Re: Measuring/Debugging XDP Performance Message-ID: <20200129130945.733deacf@carbon> In-Reply-To: References: <6c3dc8ff-e2bd-a06e-d9f0-c5be0103d266@gflclan.com> <20200123141122.3783e298@carbon> <23ec64c0-e0d7-a60b-ecc4-6ca401dc4896@gflclan.com> <20200123213808.7a1f200b@carbon> MIME-Version: 1.0 Sender: xdp-newbies-owner@vger.kernel.org List-ID: Content-Type: multipart/mixed; boundary="MP_/Hq/MD+ZN9n9isat+s6MDNry" To: Christian Deacon Cc: xdp-newbies@vger.kernel.org, brouer@redhat.com --MP_/Hq/MD+ZN9n9isat+s6MDNry Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Tue, 28 Jan 2020 12:42:21 -0600 Christian Deacon wrote: > I understand the `e1000e` driver doesn't have XDP support upstream.=20 > Therefore, I tried implementing the driver patch that adds XDP support=20 > here . Unfortunately, this patch= =20 > was based off of kernel 4.10.2 (I'm using 4.19). Therefore, I had to=20 > manually implement the patch code (this is my first time messing with=20 > NIC driver code). Sadly, it doesn't seem like the patch worked based off= =20 > of the 'perf' results which are attached. I still see "do_xdp_generic"=20 I do think it would be more valuable to implement XDP for driver igb. Given igb is a "slow" 1Gbit/s interface, we could also just fix XDP-generic to avoid the reallocation of the SKB for every packet... I've attached two patches which does exactly that. It sounds like you know howto apply patches and recompile your kernel(?) I've tested this on my testlab. The igb hardware seems is limited to 1.2Mpps. Using an XDP_DROP test program, I observe the CPU processing packets use 50% CPU time before, and 36% CPU time after the patches. --=20 Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer --MP_/Hq/MD+ZN9n9isat+s6MDNry Content-Type: application/octet-stream; name=01-04-xdp-headroom Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=01-04-xdp-headroom eGRwOiBhY2NlcHQgdGhhdCBYRFAgaGVhZHJvb20gaXNuJ3QgYWx3YXlzIGVxdWFsIFhEUF9QQUNL RVRfSEVBRFJPT00KCkZyb206IEplc3BlciBEYW5nYWFyZCBCcm91ZXIgPGJyb3VlckByZWRoYXQu Y29tPgoKVGhlIEludGVsIGJhc2VkIGRyaXZlcnMgKGl4Z2JlICsgaTQwZSkgaGF2ZSBpbXBsZW1l bnRlZCBYRFAgd2l0aApoZWFkcm9vbSAxOTIgYnl0ZXMgYW5kIG5vdCB0aGUgcmVjb21tZW5kZWQg MjU2IGJ5dGVzIGRlZmluZWQgYnkKWERQX1BBQ0tFVF9IRUFEUk9PTS4gIEZvciBnZW5lcmljLVhE UCwgYWNjZXB0IHRoYXQgdGhpcyBoZWFkcm9vbQppcyBhbHNvIGEgdmFsaWQgc2l6ZS4KClN0aWxs IGZvciBnZW5lcmljLVhEUCBpZiBoZWFkcm9vbSBpcyBsZXNzLCBzdGlsbCBleHBhbmQgaGVhZHJv b20gdG8KWERQX1BBQ0tFVF9IRUFEUk9PTSBhcyB0aGlzIGlzIHRoZSBkZWZhdWx0IGluIG1vc3Qg WERQIGRyaXZlcnMuCgpTaWduZWQtb2ZmLWJ5OiBKZXNwZXIgRGFuZ2FhcmQgQnJvdWVyIDxicm91 ZXJAcmVkaGF0LmNvbT4KLS0tCiBpbmNsdWRlL3VhcGkvbGludXgvYnBmLmggfCAgICAxICsKIG5l dC9jb3JlL2Rldi5jICAgICAgICAgICB8ICAgIDQgKystLQogMiBmaWxlcyBjaGFuZ2VkLCAzIGlu c2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvaW5jbHVkZS91YXBpL2xp bnV4L2JwZi5oIGIvaW5jbHVkZS91YXBpL2xpbnV4L2JwZi5oCmluZGV4IGYxZDc0YTJiZDIzNC4u MmQ5M2NiMWY5NDMzIDEwMDY0NAotLS0gYS9pbmNsdWRlL3VhcGkvbGludXgvYnBmLmgKKysrIGIv aW5jbHVkZS91YXBpL2xpbnV4L2JwZi5oCkBAIC0zMjkxLDYgKzMyOTEsNyBAQCBzdHJ1Y3QgYnBm X3hkcF9zb2NrIHsKIH07CiAKICNkZWZpbmUgWERQX1BBQ0tFVF9IRUFEUk9PTSAyNTYKKyNkZWZp bmUgWERQX1BBQ0tFVF9IRUFEUk9PTV9NSU4gMTkyCiAKIC8qIFVzZXIgcmV0dXJuIGNvZGVzIGZv ciBYRFAgcHJvZyB0eXBlLgogICogQSB2YWxpZCBYRFAgcHJvZ3JhbSBtdXN0IHJldHVybiBvbmUg b2YgdGhlc2UgZGVmaW5lZCB2YWx1ZXMuIEFsbCBvdGhlcgpkaWZmIC0tZ2l0IGEvbmV0L2NvcmUv ZGV2LmMgYi9uZXQvY29yZS9kZXYuYwppbmRleCAzOGJjMzVkYTM5ZjcuLmM1NGVkOTA2NjcyNCAx MDA2NDQKLS0tIGEvbmV0L2NvcmUvZGV2LmMKKysrIGIvbmV0L2NvcmUvZGV2LmMKQEAgLTQ1MzEs MTEgKzQ1MzEsMTEgQEAgc3RhdGljIHUzMiBuZXRpZl9yZWNlaXZlX2dlbmVyaWNfeGRwKHN0cnVj dCBza19idWZmICpza2IsCiAJCXJldHVybiBYRFBfUEFTUzsKIAogCS8qIFhEUCBwYWNrZXRzIG11 c3QgYmUgbGluZWFyIGFuZCBtdXN0IGhhdmUgc3VmZmljaWVudCBoZWFkcm9vbQotCSAqIG9mIFhE UF9QQUNLRVRfSEVBRFJPT00gYnl0ZXMuIFRoaXMgaXMgdGhlIGd1YXJhbnRlZSB0aGF0IGFsc28K KwkgKiBvZiBYRFBfUEFDS0VUX0hFQURST09NX01JTiBieXRlcy4gVGhpcyBpcyB0aGUgZ3VhcmFu dGVlIHRoYXQgYWxzbwogCSAqIG5hdGl2ZSBYRFAgcHJvdmlkZXMsIHRodXMgd2UgbmVlZCB0byBk byBpdCBoZXJlIGFzIHdlbGwuCiAJICovCiAJaWYgKHNrYl9pc19ub25saW5lYXIoc2tiKSB8fAot CSAgICBza2JfaGVhZHJvb20oc2tiKSA8IFhEUF9QQUNLRVRfSEVBRFJPT00pIHsKKwkgICAgc2ti X2hlYWRyb29tKHNrYikgPCBYRFBfUEFDS0VUX0hFQURST09NX01JTikgewogCQlpbnQgaHJvb20g PSBYRFBfUEFDS0VUX0hFQURST09NIC0gc2tiX2hlYWRyb29tKHNrYik7CiAJCWludCB0cm9vbSA9 IHNrYi0+dGFpbCArIHNrYi0+ZGF0YV9sZW4gLSBza2ItPmVuZDsKIAo= --MP_/Hq/MD+ZN9n9isat+s6MDNry Content-Type: application/octet-stream; name=02-igb-adjust-headroom Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=02-igb-adjust-headroom aWdiOiBpbmNyZWFzZSBoZWFkcm9vbSB0byBjb29wZXJhdGUgd2l0aCBYRFAtZ2VuZXJpYwoKRnJv bTogSmVzcGVyIERhbmdhYXJkIEJyb3VlciA8YnJvdWVyQHJlZGhhdC5jb20+CgpJbmNyZWFzZSB0 aGUgcGFja2V0IGhlYWRyb29tIHRvIDE5MiBieXRlcy4KClRoaXMgd29ya3MsIGJ1dCBkdWUgdG8g TklDIFRpbWVTdGFtcCBoZWFkZXIgKElHQl9UU19IRFJfTEVOID0gMTYpLApwcm92aWRpbmcgMTky IGJ5dGVzIGhlYWRyb29tLCBjYXVzZSB0aGUgZHJpdmVyIHRvIHVzZSBvcmRlci0xIHBhZ2VzCig4 MTkyIGJ5dGVzKSB0byBzdG9yZSBwYWNrZXRzLiAgVGhpcyBpcyBvbmx5IGEgcHJvYmxlbSBpZiB0 aGUgVFMKaGVhZGVyIGlzIGNvbWJpbmVkIHdpdGggYSBWTEFOIGhlYWRlciBhbmQgaXMgYSBNVFUg c2l6ZWQgcGFja2V0cy4KCkEgc29sdXRpb24gd291bGQgYmUgdG8gZGV0ZWN0IGlmIEhXIHRpbWVz dGFtcCBpcyBlbmFibGVkLCBidXQgSQpjb3VsZG4ndCBmaW5kIGFuIGVhc3kgdGVzdC4KClNpZ25l ZC1vZmYtYnk6IEplc3BlciBEYW5nYWFyZCBCcm91ZXIgPGJyb3VlckByZWRoYXQuY29tPgotLS0K IGRyaXZlcnMvbmV0L2V0aGVybmV0L2ludGVsL2lnYi9pZ2IuaCAgICAgIHwgICAgOSArKysrKysr LS0KIGRyaXZlcnMvbmV0L2V0aGVybmV0L2ludGVsL2lnYi9pZ2JfbWFpbi5jIHwgICAgMyArKy0K IDIgZmlsZXMgY2hhbmdlZCwgOSBpbnNlcnRpb25zKCspLCAzIGRlbGV0aW9ucygtKQoKZGlmZiAt LWdpdCBhL2RyaXZlcnMvbmV0L2V0aGVybmV0L2ludGVsL2lnYi9pZ2IuaCBiL2RyaXZlcnMvbmV0 L2V0aGVybmV0L2ludGVsL2lnYi9pZ2IuaAppbmRleCA0OWI1ZmE5ZDQ3ODMuLjRjNjg5ZmU5NDdk MSAxMDA2NDQKLS0tIGEvZHJpdmVycy9uZXQvZXRoZXJuZXQvaW50ZWwvaWdiL2lnYi5oCisrKyBi L2RyaXZlcnMvbmV0L2V0aGVybmV0L2ludGVsL2lnYi9pZ2IuaApAQCAtMTM3LDEwICsxMzcsMTUg QEAgc3RydWN0IHZmX21hY19maWx0ZXIgewogI2RlZmluZSBJR0JfUlhfSERSX0xFTgkJSUdCX1JY QlVGRkVSXzI1NgogI2RlZmluZSBJR0JfVFNfSERSX0xFTgkJMTYKIAotI2RlZmluZSBJR0JfU0tC X1BBRAkJKE5FVF9TS0JfUEFEICsgTkVUX0lQX0FMSUdOKQorLyogSGVhZHJvb20gMTkyIGZpdCBN VFUgc2l6ZSBFdGhlcm5ldCBmcmFtZSBpbnRvIDJLIHdpdGggc2tiX3NoYXJlZF9pbmZvICovCisj ZGVmaW5lIElHQl9TS0JfUEFECQkxOTIKKy8vIEZJWE1FOiBXaGF0IGFib3V0IE5FVF9JUF9BTElH Tj8KKwogI2lmIChQQUdFX1NJWkUgPCA4MTkyKQogI2RlZmluZSBJR0JfTUFYX0ZSQU1FX0JVSUxE X1NLQiBcCiAJKFNLQl9XSVRIX09WRVJIRUFEKElHQl9SWEJVRkZFUl8yMDQ4KSAtIElHQl9TS0Jf UEFEIC0gSUdCX1RTX0hEUl9MRU4pCisJIC8vIEhtbS4uLiB0aGlzIC1JR0JfVFNfSERSX0xFTiBt aWdodCBiZSBhIHByb2JsZW0KKwkvLyAyMDQ4IC0gMzIwIC0gMTkyIC0gMTYgPSAxNTIwCiAjZWxz ZQogI2RlZmluZSBJR0JfTUFYX0ZSQU1FX0JVSUxEX1NLQiAoSUdCX1JYQlVGRkVSXzIwNDggLSBJ R0JfVFNfSERSX0xFTikKICNlbmRpZgpAQCAtMzQ4LDcgKzM1Myw3IEBAIHN0YXRpYyBpbmxpbmUg dW5zaWduZWQgaW50IGlnYl9yeF9wZ19vcmRlcihzdHJ1Y3QgaWdiX3JpbmcgKnJpbmcpCiB7CiAj aWYgKFBBR0VfU0laRSA8IDgxOTIpCiAJaWYgKHJpbmdfdXNlc19sYXJnZV9idWZmZXIocmluZykp Ci0JCXJldHVybiAxOworCQlyZXR1cm4gMTsgLy8gcGFnZSBvcmRlci0xID0gODE5MiBieXRlcwog I2VuZGlmCiAJcmV0dXJuIDA7CiB9CmRpZmYgLS1naXQgYS9kcml2ZXJzL25ldC9ldGhlcm5ldC9p bnRlbC9pZ2IvaWdiX21haW4uYyBiL2RyaXZlcnMvbmV0L2V0aGVybmV0L2ludGVsL2lnYi9pZ2Jf bWFpbi5jCmluZGV4IGI0NmJmZjhmZTA1Ni4uOGVkM2RiMjdkNGQyIDEwMDY0NAotLS0gYS9kcml2 ZXJzL25ldC9ldGhlcm5ldC9pbnRlbC9pZ2IvaWdiX21haW4uYworKysgYi9kcml2ZXJzL25ldC9l dGhlcm5ldC9pbnRlbC9pZ2IvaWdiX21haW4uYwpAQCAtNDU2Nyw3ICs0NTY3LDcgQEAgc3RhdGlj IHZvaWQgaWdiX3NldF9yeF9idWZmZXJfbGVuKHN0cnVjdCBpZ2JfYWRhcHRlciAqYWRhcHRlciwK ICNpZiAoUEFHRV9TSVpFIDwgODE5MikKIAlpZiAoYWRhcHRlci0+bWF4X2ZyYW1lX3NpemUgPD0g SUdCX01BWF9GUkFNRV9CVUlMRF9TS0IpCiAJCXJldHVybjsKLQorCS8vIElzc3VlICgxNTIyIDw9 IDE1MjApIC0tPiBzZXRfcmluZ191c2VzX2xhcmdlX2J1ZmZlcgogCXNldF9yaW5nX3VzZXNfbGFy Z2VfYnVmZmVyKHJ4X3JpbmcpOwogI2VuZGlmCiB9CkBAIC02MjU2LDYgKzYyNTYsNyBAQCBzdGF0 aWMgaW50IGlnYl9jaGFuZ2VfbXR1KHN0cnVjdCBuZXRfZGV2aWNlICpuZXRkZXYsIGludCBuZXdf bXR1KQogewogCXN0cnVjdCBpZ2JfYWRhcHRlciAqYWRhcHRlciA9IG5ldGRldl9wcml2KG5ldGRl dik7CiAJaW50IG1heF9mcmFtZSA9IG5ld19tdHUgKyBFVEhfSExFTiArIEVUSF9GQ1NfTEVOICsg VkxBTl9ITEVOOworCS8vIDE1MDAgKyAxNCArIDQgKyA0ID0gMTUyMgogCiAJLyogYWRqdXN0IG1h eCBmcmFtZSB0byBiZSBhdCBsZWFzdCB0aGUgc2l6ZSBvZiBhIHN0YW5kYXJkIGZyYW1lICovCiAJ aWYgKG1heF9mcmFtZSA8IChFVEhfRlJBTUVfTEVOICsgRVRIX0ZDU19MRU4pKQo= --MP_/Hq/MD+ZN9n9isat+s6MDNry--