From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DA84C10F13 for ; Sun, 14 Apr 2019 20:40:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0CFB720850 for ; Sun, 14 Apr 2019 20:40:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qpSJuRQu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727151AbfDNUke (ORCPT ); Sun, 14 Apr 2019 16:40:34 -0400 Received: from mail-it1-f170.google.com ([209.85.166.170]:51745 "EHLO mail-it1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbfDNUke (ORCPT ); Sun, 14 Apr 2019 16:40:34 -0400 Received: by mail-it1-f170.google.com with SMTP id s3so9168677itk.1 for ; Sun, 14 Apr 2019 13:40:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=nbTcNJNOatskYKNLAtPh9amzfh8PP0HYAJn4HWyaVis=; b=qpSJuRQuTjMhelEwy7MybajO2+odxBdralOQPqC8OrQnlRYkSqTxESQGJFeTQdAY5k 4aYpn2qJWQyqZkESAixHA4Oq+WiuMmvZo6Dmg+5VDcZwGlKhRtp6rHJJhHnli1/NmUgw sggz0RKmVLWqPOy9PmJb3qxNcdCgLMIqT5ovtd/fBAmp6YVhcsksaGErx4ihu1qMhqgM gY2olKmu/DVT6F8gJRJgleNw7bASjpJZOyKRM9aJMpKlvhmWUGfFOJtk3M0sOc6T2fkz 8kSoGtfe+xFtwXpNVsp1Vy17/JmM+CG9+fh6ixa9WoXtyWIF85IrN2sMJik46Vd7cXej 8iQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=nbTcNJNOatskYKNLAtPh9amzfh8PP0HYAJn4HWyaVis=; b=h3CvwT9Z+QGRhC0rtK9II39RxwNrlqYv1qzkcQ9kMAkhUuiY/XRRqbCN2kQB76jXJ2 crzvZOLDqVmkusopCsUbGECuLc4sPCRYs2qEfnl+pKWBcIRDmKo8C+sYnuzM6WWGCWdR bffLmVDRQDOM8TiIbnCD4MMqL4ioP8KleUZ4qoTiucaTpdzSsI1JFYD72x9H7YBNMqee myGE7H3OOngYZWhy0TRXxmFryVwcbWc/iZ4UYyWdMLhmdim2SEGcZGc/9UuBpfcUsJvb bYhnIELZ/Tmm2Il8VgUyHTwMT3yC6QmblAo72UOB1CeOaxns0lGD3gWtUJaoy94UxrKW S/Wg== X-Gm-Message-State: APjAAAXj0QGc5STmiVjlu75uOC4vUlxicoeilZi8Ks/ActJL9eBM/j2c RsnFw3ajoAERlexjhRBG/FosS6N7 X-Google-Smtp-Source: APXvYqwpgGD9xbEz4nKoD48rnXcKnhGLP4HNQHlJi/3m8khtzhuW6VAF83kHgeOadokCwnYPDVlMRg== X-Received: by 2002:a24:4584:: with SMTP id c4mr22899982itd.163.1555274433308; Sun, 14 Apr 2019 13:40:33 -0700 (PDT) Received: from [192.168.86.250] ([184.63.162.180]) by smtp.gmail.com with ESMTPSA id c202sm7630336itc.30.2019.04.14.13.40.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Apr 2019 13:40:32 -0700 (PDT) From: John Fastabend Subject: Re: kTLS broken somewhere between 4.18 and 5.0 To: Andre Tomt , "Steinar H. Gunderson" , netdev@vger.kernel.org, Daniel Borkmann References: <20190413153435.73ecztpnbivrckvj@sesse.net> <4cdfca7f-bee0-56f5-e512-8ad2e4e6dfcf@tomt.net> Message-ID: Date: Sun, 14 Apr 2019 13:40:20 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <4cdfca7f-bee0-56f5-e512-8ad2e4e6dfcf@tomt.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 4/13/19 6:56 PM, Andre Tomt wrote: > On 13.04.2019 17:34, Steinar H. Gunderson wrote: >> Hi, >> >> I've been using kTLS for a while, with my video reflector Cubemap >> (https://git.sesse.net/?p=cubemap). After I upgraded my server from >> 4.18.11 to 5.0.6, seemingly I've started seeing corruption. The data sent >> with send() (HTTP headers, HLS playlists) appears to be fine, but sendfile() >> (actual video data, from a file on tmpfs) is not; after ~20 kB of data >> (19626 in one test here), the data appears to be randomly corrupted. Diffing >> non-TLS (good) and TLS (bad) video data: >> >>    00004c70: fa 70 c5 71 b5 f5 b7 ac 74 b0 ca 80 02 4c 06 3f  .p.q....t....L.? >>    00004c80: 5c 5b 0c b3 e0 a0 c3 21 93 d3 6e 65 36 70 0a 27  \[.....!..ne6p.' >>    00004c90: 84 67 16 2c 95 c0 55 e1 04 76 52 10 50 5d 00 26  .g.,..U..vR.P].& >> >>   -00004ca0: 0c b8 84 70 7e ed 12 8f 5e 7e 18 c0 06 20 02 54  ...p~...^~... .T >>   +00004ca0: 0c b8 84 70 7e ed 12 8f 5e 7e 0a 60 9f 1f 97 f2  ...p~...^~.`.... >> >>   -00004cb0: 1e 4c c1 71 7d 0b 91 28 23 98 09 ae c4 95 ae 7f  .L.q}..(#....... >>   +00004cb0: 6e 17 50 03 67 fa 2f 83 b0 88 eb fc 54 f2 0b 00  n.P.g./.....T... >> >>   -00004cc0: a2 92 20 b8 f2 b6 72 2a e8 7e d7 27 99 65 56 70  .. ...r*.~.'.eVp >>   +00004cc0: 6c 9e a1 02 b4 30 11 25 d7 58 b0 0c c0 6c e1 bd  l....0.%.X...l.. >> >> It never appears to get back into sync after that. Interestingly, it is >> _consistently_ wrong; if I download the same fragment multiple times, it >> breaks at the same place and gives the same garbage (but different fragments >> give different divergence points). Tested with both wget and Chrome. >> Does anyone know what could be wrong? >> >> (It is, unfortunately, not easy for me to reboot this server at will, so a >> bisect could be hard.) >> >> Please Cc me on any replies, I'm not subscribed to netdev. >> >> /* Steinar */ >> > > > Reproduced and bisected, the problem showed up in v4.20-rc1. Unfortunately the commit seems to have some significant dependencies so I was unable to verify by reverting it on 4.20. > Hi thanks I'll take a look this evening or first thing tomorrow. I have a couple other fixes queued up for ktls as well so I'll see if we can get a fix for this included in that series. Thanks! John