From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CE51C4741F for ; Sun, 8 Nov 2020 14:47:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4910C206F4 for ; Sun, 8 Nov 2020 14:47:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="z+sE+J+N"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NEe+aXTf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4910C206F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BVUq+IKHV4UiP58dklC5Be92ADNwQh73UzLMsG0o0eQ=; b=z+sE+J+N6GbmETpPW7ElFlQbb 03nrt7gXgFciRbLfofgAnXxtG3V5X4ff+Aq/TqK+P/N9TEjE3CxiIcdPc041wsGj9hyFOQWElHv6A ghDln6cUUNOKhJgco1Y4gjh51op0wcGFvFoucoAqLGfePHlg/uAYre7VHvUvFI1OW6f/WYg0wXEc2 C3xRadcoyH96LK5YloouEqUWmKyDFru8B8d4vHf9iALtRMJ3FeByytSi5uohlxMXkZFvcidAyo6sB Gf1o1dfnBc+lZ5QQNBb+MBWhfNWhQa7mCPz348Yr4VlpiTPqoq9Z5TqHCfV0E0jQhKHPyQrxkZBuF Rx+ElJ/8A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kbly0-0003Xh-QG; Sun, 08 Nov 2020 14:46:52 +0000 Received: from mail-ed1-x544.google.com ([2a00:1450:4864:20::544]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kblxy-0003X8-3o for linux-nvme@lists.infradead.org; Sun, 08 Nov 2020 14:46:51 +0000 Received: by mail-ed1-x544.google.com with SMTP id q3so6019056edr.12 for ; Sun, 08 Nov 2020 06:46:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=vnFceaW+8mscDH815HlhItngZTzRyIa8sj9BZAwWtU4=; b=NEe+aXTfKYtiWMOLmyxRfSnDiUe3M9HiVtbtSwYPZX9/U4BgK73c0jPQHdiH1lWduX q6IYYhwYvdtpeJohRkzD4c1v/3VPGL9j04Gcb0QU/g2HhxGF/sqZAjZIaZXOp6kuy7LE kfMCd6EzNwX+Ljpoz0KKdfds8MR/PQeeQeH+87BGnDqhMgnuZUpagMXa2YC6W/W/ZHT9 1JgbjiIx0hPoADyDlMw8giXnGdjIqIbq+2k7KUfevO2daa+nOE2x5DgMd3GUDbkPTcKA tDuQ5yeuvpIP/LEd1KSCijmPw+jiVIwJ0FCK1fuXg4GjKHevQ6tVJbG7n2Hepr0e3ZaH Omjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=vnFceaW+8mscDH815HlhItngZTzRyIa8sj9BZAwWtU4=; b=Nhjy7lbatkYbO/3VIugfTWGHy9z0e5ub4DnW8iuVTUoNuZ7oiCHgTIYxLIhvx9iTaB LgL2Q1XjlCiXzdFs7Bt/EvHctsaWdbE/egmacdW0BC4kYwspqtfGg9BxtGtypIysQAU0 Zi79ahfFrfxppEJa/VUiw4kTdil8LKoAzJ10mQ3Wb3RTgAMhdOYpzaqsjJB2y+7la5KR PPBt6ibVb/CEeaA4/exiF5OTm75ED1DRfPf0gFB+bVy7zfjssHPlPPpFVH8SaGOtVFcw FWc/Sqn47ygiJVtR4nA12Bmre/VdvXjYvqxd6ZHGraufiyFkVIjsJBtMZk3tKtExpPeW AAgg== X-Gm-Message-State: AOAM532bES1p0qV9RioFCq0THhUj7hTLuUiUM4bb8o2+9DeXFT8LCqyk cGgJQWd5kxmyIhcb6hlp/z+wQ3TAQJwbLg== X-Google-Smtp-Source: ABdhPJxE+C6ozFbnRWrF6XkNxbrPq9f0cFvseaXfeT9nhnfutGl6IR1ck7ZMXXn+1R+f9M/CX6lJUQ== X-Received: by 2002:a50:be8f:: with SMTP id b15mr10870511edk.180.1604846808791; Sun, 08 Nov 2020 06:46:48 -0800 (PST) Received: from [132.68.43.131] ([132.68.43.131]) by smtp.gmail.com with ESMTPSA id o20sm6144212eja.34.2020.11.08.06.46.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 08 Nov 2020 06:46:48 -0800 (PST) Subject: Re: [PATCH net-next RFC v1 07/10] nvme-tcp : Recalculate crc in the end of the capsule To: Sagi Grimberg , Boris Pismenny , kuba@kernel.org, davem@davemloft.net, saeedm@nvidia.com, hch@lst.de, axboe@fb.com, kbusch@kernel.org, viro@zeniv.linux.org.uk, edumazet@google.com References: <20200930162010.21610-1-borisp@mellanox.com> <20200930162010.21610-8-borisp@mellanox.com> From: Boris Pismenny Message-ID: Date: Sun, 8 Nov 2020 16:46:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201108_094650_201719_FC5BCD95 X-CRM114-Status: GOOD ( 21.99 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yoray Zack , Ben Ben-Ishay , boris.pismenny@gmail.com, linux-nvme@lists.infradead.org, netdev@vger.kernel.org, Or Gerlitz Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org Ck9uIDA5LzEwLzIwMjAgMTo0NCwgU2FnaSBHcmltYmVyZyB3cm90ZToKPj4gY3JjIG9mZmxvYWQg b2YgdGhlIG52bWUgY2Fwc3VsZS4gQ2hlY2sgaWYgYWxsIHRoZSBza2IgYml0cwo+PiBhcmUgb24s IGFuZCBpZiBub3QgcmVjYWxjdWxhdGUgdGhlIGNyYyBpbiBTVyBhbmQgY2hlY2sgaXQuCj4gQ2Fu IHlvdSBjbGFyaWZ5IGluIHRoZSBwYXRjaCBkZXNjcmlwdGlvbiB0aGF0IHRoaXMgaXMgb25seQo+ IGZvciBwZHUgZGF0YSBkaWdlc3QgYW5kIG5vdCBoZWFkZXIgZGlnZXN0PwoKV2lsbCBkbwoKPgo+ PiBUaGlzIHBhdGNoIHJld29ya3MgdGhlIHJlY2VpdmUtc2lkZSBjcmMgY2FsY3VsYXRpb24gdG8g YWx3YXlzCj4+IHJ1biBhdCB0aGUgZW5kLCBzbyBhcyB0byBrZWVwIGEgc2luZ2xlIGZsb3cgZm9y IGJvdGggb2ZmbG9hZAo+PiBhbmQgbm9uLW9mZmxvYWQuIFRoaXMgY2hhbmdlIHNpbXBsaWZpZXMg dGhlIGNvZGUsIGJ1dCBpdCBtYXkgZGVncmFkZQo+PiBwZXJmb3JtYW5jZSBmb3Igbm9uLW9mZmxv YWQgY3JjIGNhbGN1bGF0aW9uLgo+ID8/Cj4KPiAgRnJvbSBteSBzY2FuIGl0IGRvZWVzbid0IGxv b2sgbGlrZSB5b3UgZG8gdGhhdC4uIEFtIEkgbWlzc2luZyBzb21ldGhpbmc/Cj4gQ2FuIHlvdSBl eHBsYWluPwoKVGhlIHBlcmZvcm1hbmNlIG9mIENSQyBkYXRhIGRpZ2VzdCBpbiB0aGUgb2ZmbG9h ZCdzIGZhbGxiYWNrIHBhdGggbWF5IGJlIGxlc3MgZ29vZCBjb21wYXJlZCB0byBDUkMgY2FsY3Vs YXRpb24gd2l0aCBza2JfY29weV9hbmRfaGFzaC4KVG8gYmUgY2xlYXIsIHRoZSBmYWxsYmFjayBw YXRoIGlzIG9jY3VycyB3aGVuIGBxdWV1ZS0+ZGF0YV9kaWdlc3QgJiYgdGVzdF9iaXQoTlZNRV9U Q1BfUV9PRkZfQ1JDX1JYLCAmcXVldWUtPmZsYWdzKWAsIHdoaWxlIHdlIHJlY2VpdmUgU0tCcyB3 aGVyZSBgc2tiLT5kZHBfY3JjID0gMGAKCj4KPj4gICAJcnEgPSBibGtfbXFfdGFnX3RvX3JxKG52 bWVfdGNwX3RhZ3NldChxdWV1ZSksIHBkdS0+Y29tbWFuZF9pZCk7Cj4+ICAgCWlmICghcnEpIHsK Pj4gICAJCWRldl9lcnIocXVldWUtPmN0cmwtPmN0cmwuZGV2aWNlLAo+PiBAQCAtOTkyLDcgKzEw MzEsNyBAQCBzdGF0aWMgaW50IG52bWVfdGNwX3JlY3ZfZGF0YShzdHJ1Y3QgbnZtZV90Y3BfcXVl dWUgKnF1ZXVlLCBzdHJ1Y3Qgc2tfYnVmZiAqc2tiLAo+PiAgIAkJcmVjdl9sZW4gPSBtaW5fdChz aXplX3QsIHJlY3ZfbGVuLAo+PiAgIAkJCQlpb3ZfaXRlcl9jb3VudCgmcmVxLT5pdGVyKSk7Cj4+ ICAgCj4+IC0JCWlmIChxdWV1ZS0+ZGF0YV9kaWdlc3QpCj4+ICsJCWlmIChxdWV1ZS0+ZGF0YV9k aWdlc3QgJiYgIXRlc3RfYml0KE5WTUVfVENQX1FfT0ZGTE9BRFMsICZxdWV1ZS0+ZmxhZ3MpKQo+ PiAgIAkJCXJldCA9IHNrYl9jb3B5X2FuZF9oYXNoX2RhdGFncmFtX2l0ZXIoc2tiLCAqb2Zmc2V0 LAo+PiAgIAkJCQkmcmVxLT5pdGVyLCByZWN2X2xlbiwgcXVldWUtPnJjdl9oYXNoKTsKPiBUaGlz IGlzIHRoZSBza2IgY29weSBhbmQgaGFzaCwgbm90IGNsZWFyIHdoeSB5b3Ugc2F5IHRoYXQgeW91 IG1vdmUgdGhpcwo+IHRvIHRoZSBlbmQuLi4KClNlZSB0aGUgb2ZmbG9hZCBmYWxsYmFjayBwYXRo IGJlbG93Cgo+Cj4+ICAgCQllbHNlCj4+IEBAIC0xMDEyLDcgKzEwNTEsNiBAQCBzdGF0aWMgaW50 IG52bWVfdGNwX3JlY3ZfZGF0YShzdHJ1Y3QgbnZtZV90Y3BfcXVldWUgKnF1ZXVlLCBzdHJ1Y3Qg c2tfYnVmZiAqc2tiLAo+PiAgIAo+PiAgIAlpZiAoIXF1ZXVlLT5kYXRhX3JlbWFpbmluZykgewo+ PiAgIAkJaWYgKHF1ZXVlLT5kYXRhX2RpZ2VzdCkgewo+PiAtCQkJbnZtZV90Y3BfZGRnc3RfZmlu YWwocXVldWUtPnJjdl9oYXNoLCAmcXVldWUtPmV4cF9kZGdzdCk7Cj4gSWYgSSBpbnN0ZWFkIGRv Ogo+IAkJCWlmICghdGVzdF9iaXQoTlZNRV9UQ1BfUV9PRkZMT0FEUywKPiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAmcXVldWUtPmZsYWdzKSkKPiAJCQkJbnZtZV90Y3Bf ZGRnc3RfZmluYWwocXVldWUtPnJjdl9oYXNoLAo+IAkJCQkJCSAgICAgJnF1ZXVlLT5leHBfZGRn c3QpOwo+Cj4gRG9lcyB0aGF0IGhlbHAgdGhlIG1lc3MgaW4gbnZtZV90Y3BfcmVjdl9kZGdzdD8K Ck5vdCByZWFsbHksIGFzIHRoZSBjb2RlIHBhdGggdGhlcmUgdGFrZXMgY2FyZSBvZiB0aGUgZmFs bGJhY2sgcGF0aCwgaS5lLiBvZmZsb2FkZWQgcmVxdWVzdGVkLCBidXQgZGlkbid0IHN1Y2NlZWQu Cgo+Cj4+ICAgCQkJcXVldWUtPmRkZ3N0X3JlbWFpbmluZyA9IE5WTUVfVENQX0RJR0VTVF9MRU5H VEg7Cj4+ICAgCQl9IGVsc2Ugewo+PiAgIAkJCWlmIChwZHUtPmhkci5mbGFncyAmIE5WTUVfVENQ X0ZfREFUQV9TVUNDRVNTKSB7Cj4+IEBAIC0xMDMzLDggKzEwNzEsMTEgQEAgc3RhdGljIGludCBu dm1lX3RjcF9yZWN2X2RkZ3N0KHN0cnVjdCBudm1lX3RjcF9xdWV1ZSAqcXVldWUsCj4+ICAgCWNo YXIgKmRkZ3N0ID0gKGNoYXIgKikmcXVldWUtPnJlY3ZfZGRnc3Q7Cj4+ICAgCXNpemVfdCByZWN2 X2xlbiA9IG1pbl90KHNpemVfdCwgKmxlbiwgcXVldWUtPmRkZ3N0X3JlbWFpbmluZyk7Cj4+ICAg CW9mZl90IG9mZiA9IE5WTUVfVENQX0RJR0VTVF9MRU5HVEggLSBxdWV1ZS0+ZGRnc3RfcmVtYWlu aW5nOwo+PiArCWJvb2wgZGRnc3Rfb2ZmbG9hZF9mYWlsOwo+PiAgIAlpbnQgcmV0Owo+PiAgIAo+ PiArCWlmICh0ZXN0X2JpdChOVk1FX1RDUF9RX09GRkxPQURTLCAmcXVldWUtPmZsYWdzKSkKPj4g KwkJbnZtZV90Y3BfZGV2aWNlX2RkZ3N0X3VwZGF0ZShxdWV1ZSwgc2tiKTsKPj4gICAJcmV0ID0g c2tiX2NvcHlfYml0cyhza2IsICpvZmZzZXQsICZkZGdzdFtvZmZdLCByZWN2X2xlbik7Cj4+ICAg CWlmICh1bmxpa2VseShyZXQpKQo+PiAgIAkJcmV0dXJuIHJldDsKPj4gQEAgLTEwNDUsMTIgKzEw ODYsMjEgQEAgc3RhdGljIGludCBudm1lX3RjcF9yZWN2X2RkZ3N0KHN0cnVjdCBudm1lX3RjcF9x dWV1ZSAqcXVldWUsCj4+ICAgCWlmIChxdWV1ZS0+ZGRnc3RfcmVtYWluaW5nKQo+PiAgIAkJcmV0 dXJuIDA7Cj4+ICAgCj4+IC0JaWYgKHF1ZXVlLT5yZWN2X2RkZ3N0ICE9IHF1ZXVlLT5leHBfZGRn c3QpIHsKPj4gLQkJZGV2X2VycihxdWV1ZS0+Y3RybC0+Y3RybC5kZXZpY2UsCj4+IC0JCQkiZGF0 YSBkaWdlc3QgZXJyb3I6IHJlY3YgJSN4IGV4cGVjdGVkICUjeFxuIiwKPj4gLQkJCWxlMzJfdG9f Y3B1KHF1ZXVlLT5yZWN2X2RkZ3N0KSwKPj4gLQkJCWxlMzJfdG9fY3B1KHF1ZXVlLT5leHBfZGRn c3QpKTsKPj4gLQkJcmV0dXJuIC1FSU87Cj4+ICsJZGRnc3Rfb2ZmbG9hZF9mYWlsID0gIW52bWVf dGNwX2RldmljZV9kZGdzdF9vayhxdWV1ZSk7Cj4+ICsJaWYgKCF0ZXN0X2JpdChOVk1FX1RDUF9R X09GRkxPQURTLCAmcXVldWUtPmZsYWdzKSB8fAo+PiArCSAgICBkZGdzdF9vZmZsb2FkX2ZhaWwp IHsKPj4gKwkJaWYgKHRlc3RfYml0KE5WTUVfVENQX1FfT0ZGTE9BRFMsICZxdWV1ZS0+ZmxhZ3Mp ICYmCj4+ICsJCSAgICBkZGdzdF9vZmZsb2FkX2ZhaWwpCj4+ICsJCQludm1lX3RjcF9jcmNfcmVj YWxjdWxhdGUocXVldWUsIHBkdSk7Cj4+ICsKPj4gKwkJbnZtZV90Y3BfZGRnc3RfZmluYWwocXVl dWUtPnJjdl9oYXNoLCAmcXVldWUtPmV4cF9kZGdzdCk7Cj4+ICsJCWlmIChxdWV1ZS0+cmVjdl9k ZGdzdCAhPSBxdWV1ZS0+ZXhwX2RkZ3N0KSB7Cj4+ICsJCQlkZXZfZXJyKHF1ZXVlLT5jdHJsLT5j dHJsLmRldmljZSwKPj4gKwkJCQkiZGF0YSBkaWdlc3QgZXJyb3I6IHJlY3YgJSN4IGV4cGVjdGVk ICUjeFxuIiwKPj4gKwkJCQlsZTMyX3RvX2NwdShxdWV1ZS0+cmVjdl9kZGdzdCksCj4+ICsJCQkJ bGUzMl90b19jcHUocXVldWUtPmV4cF9kZGdzdCkpOwo+PiArCQkJcmV0dXJuIC1FSU87Cj4gVGhp cyBnZXRzIGNvbnZvbHV0ZWQgaGVyZS4uLgoKV2lsbCB0cnkgdG8gc2ltcGxpZnksIHRoZSBnZW5l cmFsIGlkZWEgaXMgdGhhdCB0aGVyZSBhcmUgMyBwYXRocyB3aXRoIGNvbW1vbiBjb2RlOgoxLiBu b24tb2ZmbG9hZAoyLiBvZmZsb2FkIGZhaWxlZAozLiBvZmZsb2FkIHN1Y2Nlc3MKKDEpIGFuZCAo Mikgc2hhcmUgdGhlIGNvZGUgZm9yIGZpbmFsaXppbmcgY2hlY2tpbmcgdGhlIGRhdGEgZGlnZXN0 LCB3aGlsZSAoMykgc2tpcHMgdGhpcyBlbnRpcmVseS4KCkluIG90aGVyIHdvcmRzLCBob3cgYWJv dXQgdGhpczoKYGBgCsKgwqDCoMKgwqDCoMKgwqDCoCBvZmZsb2FkX2ZhaWwgPSAhbnZtZV90Y3Bf ZGRwX2RkZ3N0X29rKHF1ZXVlKTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg CsKgwqDCoMKgwqDCoMKgwqDCoCBvZmZsb2FkID0gdGVzdF9iaXQoTlZNRV9UQ1BfUV9PRkZfQ1JD X1JYLCAmcXVldWUtPmZsYWdzKTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKgwqDCoMKgwqDCoMKgwqDCoCBpZiAo IW9mZmxvYWQgfHwgb2ZmbG9hZF9mYWlsKSB7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAKwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCBpZiAob2ZmbG9hZCAmJiBvZmZsb2FkX2ZhaWwpIC8vIHNvZnR3YXJlLWZh bGxiYWNrwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoArCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCBudm1lX3RjcF9kZHBfZGRnc3RfcmVjYWxjKHF1ZXVlLCBwZHUpO8KgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAKwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgbnZtZV90Y3BfZGRnc3RfZmluYWwocXVldWUtPnJj dl9oYXNoLCAmcXVldWUtPmV4cF9kZGdzdCk7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgaWYgKHF1ZXVlLT5yZWN2X2RkZ3N0ICE9IHF1ZXVlLT5leHBfZGRnc3QpIHvCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoArCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCBkZXZfZXJyKHF1ZXVlLT5jdHJsLT5jdHJsLmRldmljZSzCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoArCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgImRhdGEgZGlnZXN0IGVycm9yOiByZWN2ICUjeCBleHBl Y3RlZCAlI3hcbiIswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oArCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgbGUzMl90b19jcHUocXVldWUtPnJlY3ZfZGRnc3QpLMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgCsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCBsZTMyX3RvX2NwdShxdWV1ZS0+ZXhwX2RkZ3N0KSk7wqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAKwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg cmV0dXJuIC1FSU87wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAKwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCB9wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oArCoMKgwqDCoMKgwqDCoMKgwqAgfcKgCmBgYAoKPgo+PiArCQl9Cj4+ICAgCX0KPj4gICAKPj4g ICAJaWYgKHBkdS0+aGRyLmZsYWdzICYgTlZNRV9UQ1BfRl9EQVRBX1NVQ0NFU1MpIHsKPj4KCgpf X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpMaW51eC1udm1l IG1haWxpbmcgbGlzdApMaW51eC1udm1lQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3Rz LmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1udm1lCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CA3DC4741F for ; Sun, 8 Nov 2020 14:46:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E0AB7206F4 for ; Sun, 8 Nov 2020 14:46:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NEe+aXTf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728104AbgKHOqv (ORCPT ); Sun, 8 Nov 2020 09:46:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727570AbgKHOqu (ORCPT ); Sun, 8 Nov 2020 09:46:50 -0500 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 316F3C0613CF for ; Sun, 8 Nov 2020 06:46:50 -0800 (PST) Received: by mail-ed1-x542.google.com with SMTP id t11so5992641edj.13 for ; Sun, 08 Nov 2020 06:46:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=vnFceaW+8mscDH815HlhItngZTzRyIa8sj9BZAwWtU4=; b=NEe+aXTfKYtiWMOLmyxRfSnDiUe3M9HiVtbtSwYPZX9/U4BgK73c0jPQHdiH1lWduX q6IYYhwYvdtpeJohRkzD4c1v/3VPGL9j04Gcb0QU/g2HhxGF/sqZAjZIaZXOp6kuy7LE kfMCd6EzNwX+Ljpoz0KKdfds8MR/PQeeQeH+87BGnDqhMgnuZUpagMXa2YC6W/W/ZHT9 1JgbjiIx0hPoADyDlMw8giXnGdjIqIbq+2k7KUfevO2daa+nOE2x5DgMd3GUDbkPTcKA tDuQ5yeuvpIP/LEd1KSCijmPw+jiVIwJ0FCK1fuXg4GjKHevQ6tVJbG7n2Hepr0e3ZaH Omjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=vnFceaW+8mscDH815HlhItngZTzRyIa8sj9BZAwWtU4=; b=ma/7l8AqoYIGKD4Kv675hZWi+WJiPVkZ95F+/3xpbnAMshw9v/Bg2kZHXysjGTAfJm M0WiFEYHlbjROEJ5hDJtwt6V52yqAPLY4SgsIYpLm+br43gWwrvkpSc1BdKvBpTATeKT q8bFHq6SLqgXaPsWplLS4a00P5/xU6Aw3SoKXhJdp2QYf/tE6VdZLW2B+PrCZv22wMWe J7K7Nqy1SEnGvfew5JAaqGlZ1rtmxtDEpJLcrtnhDofotFJAxjayW4IWdJh/GrLe5x84 E++nbJPkvZkA9PLvD2R3Cu9IEwO3ztY2POw3BKlU44HoxcLpazE8Kcwe/MM8JlBlPxP5 do/A== X-Gm-Message-State: AOAM530FqPEPTMAG6E+bVyXbCHkPoiH/sfuWaOm5vr8vWSOp/GFauCHK hMJvA5Lpq7zTureXf4RwH74= X-Google-Smtp-Source: ABdhPJxE+C6ozFbnRWrF6XkNxbrPq9f0cFvseaXfeT9nhnfutGl6IR1ck7ZMXXn+1R+f9M/CX6lJUQ== X-Received: by 2002:a50:be8f:: with SMTP id b15mr10870511edk.180.1604846808791; Sun, 08 Nov 2020 06:46:48 -0800 (PST) Received: from [132.68.43.131] ([132.68.43.131]) by smtp.gmail.com with ESMTPSA id o20sm6144212eja.34.2020.11.08.06.46.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 08 Nov 2020 06:46:48 -0800 (PST) Subject: Re: [PATCH net-next RFC v1 07/10] nvme-tcp : Recalculate crc in the end of the capsule To: Sagi Grimberg , Boris Pismenny , kuba@kernel.org, davem@davemloft.net, saeedm@nvidia.com, hch@lst.de, axboe@fb.com, kbusch@kernel.org, viro@zeniv.linux.org.uk, edumazet@google.com Cc: Yoray Zack , Ben Ben-Ishay , boris.pismenny@gmail.com, linux-nvme@lists.infradead.org, netdev@vger.kernel.org, Or Gerlitz References: <20200930162010.21610-1-borisp@mellanox.com> <20200930162010.21610-8-borisp@mellanox.com> From: Boris Pismenny Message-ID: Date: Sun, 8 Nov 2020 16:46:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 09/10/2020 1:44, Sagi Grimberg wrote: >> crc offload of the nvme capsule. Check if all the skb bits >> are on, and if not recalculate the crc in SW and check it. > Can you clarify in the patch description that this is only > for pdu data digest and not header digest? Will do > >> This patch reworks the receive-side crc calculation to always >> run at the end, so as to keep a single flow for both offload >> and non-offload. This change simplifies the code, but it may degrade >> performance for non-offload crc calculation. > ?? > > From my scan it doeesn't look like you do that.. Am I missing something? > Can you explain? The performance of CRC data digest in the offload's fallback path may be less good compared to CRC calculation with skb_copy_and_hash. To be clear, the fallback path is occurs when `queue->data_digest && test_bit(NVME_TCP_Q_OFF_CRC_RX, &queue->flags)`, while we receive SKBs where `skb->ddp_crc = 0` > >> rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id); >> if (!rq) { >> dev_err(queue->ctrl->ctrl.device, >> @@ -992,7 +1031,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb, >> recv_len = min_t(size_t, recv_len, >> iov_iter_count(&req->iter)); >> >> - if (queue->data_digest) >> + if (queue->data_digest && !test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags)) >> ret = skb_copy_and_hash_datagram_iter(skb, *offset, >> &req->iter, recv_len, queue->rcv_hash); > This is the skb copy and hash, not clear why you say that you move this > to the end... See the offload fallback path below > >> else >> @@ -1012,7 +1051,6 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb, >> >> if (!queue->data_remaining) { >> if (queue->data_digest) { >> - nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst); > If I instead do: > if (!test_bit(NVME_TCP_Q_OFFLOADS, > &queue->flags)) > nvme_tcp_ddgst_final(queue->rcv_hash, > &queue->exp_ddgst); > > Does that help the mess in nvme_tcp_recv_ddgst? Not really, as the code path there takes care of the fallback path, i.e. offloaded requested, but didn't succeed. > >> queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH; >> } else { >> if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) { >> @@ -1033,8 +1071,11 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue, >> char *ddgst = (char *)&queue->recv_ddgst; >> size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining); >> off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining; >> + bool ddgst_offload_fail; >> int ret; >> >> + if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags)) >> + nvme_tcp_device_ddgst_update(queue, skb); >> ret = skb_copy_bits(skb, *offset, &ddgst[off], recv_len); >> if (unlikely(ret)) >> return ret; >> @@ -1045,12 +1086,21 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue, >> if (queue->ddgst_remaining) >> return 0; >> >> - if (queue->recv_ddgst != queue->exp_ddgst) { >> - dev_err(queue->ctrl->ctrl.device, >> - "data digest error: recv %#x expected %#x\n", >> - le32_to_cpu(queue->recv_ddgst), >> - le32_to_cpu(queue->exp_ddgst)); >> - return -EIO; >> + ddgst_offload_fail = !nvme_tcp_device_ddgst_ok(queue); >> + if (!test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags) || >> + ddgst_offload_fail) { >> + if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags) && >> + ddgst_offload_fail) >> + nvme_tcp_crc_recalculate(queue, pdu); >> + >> + nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst); >> + if (queue->recv_ddgst != queue->exp_ddgst) { >> + dev_err(queue->ctrl->ctrl.device, >> + "data digest error: recv %#x expected %#x\n", >> + le32_to_cpu(queue->recv_ddgst), >> + le32_to_cpu(queue->exp_ddgst)); >> + return -EIO; > This gets convoluted here... Will try to simplify, the general idea is that there are 3 paths with common code: 1. non-offload 2. offload failed 3. offload success (1) and (2) share the code for finalizing checking the data digest, while (3) skips this entirely. In other words, how about this: ```           offload_fail = !nvme_tcp_ddp_ddgst_ok(queue);                                                          offload = test_bit(NVME_TCP_Q_OFF_CRC_RX, &queue->flags);                                              if (!offload || offload_fail) {                                                                                if (offload && offload_fail) // software-fallback                                                              nvme_tcp_ddp_ddgst_recalc(queue, pdu);                                                                                                                                                                nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);                                              if (queue->recv_ddgst != queue->exp_ddgst) {                                                                   dev_err(queue->ctrl->ctrl.device,                                                                              "data digest error: recv %#x expected %#x\n",                                                          le32_to_cpu(queue->recv_ddgst),                                                                        le32_to_cpu(queue->exp_ddgst));                                                                return -EIO;                                                                                   }                                                                                              }  ``` > >> + } >> } >> >> if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) { >>