From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 559E2C282D7 for ; Wed, 30 Jan 2019 18:22:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2BE55218AC for ; Wed, 30 Jan 2019 18:22:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="zuHJX7Fn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733058AbfA3SWS (ORCPT ); Wed, 30 Jan 2019 13:22:18 -0500 Received: from mail-qt1-f196.google.com ([209.85.160.196]:42051 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732830AbfA3SWR (ORCPT ); Wed, 30 Jan 2019 13:22:17 -0500 Received: by mail-qt1-f196.google.com with SMTP id d19so565343qtq.9 for ; Wed, 30 Jan 2019 10:22:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=T03oF8duMhtzSYksLQyHmY5K2x9PKW2i5l7rywMOwFo=; b=zuHJX7FnMPoxZKVzMJBB3M2y3qNCpofXmgJNB+O5pmPD/qPY9YeuvDej1yyy3ZXSWi zBhz5B1AH1Q++a9JBgPhOjYRAaxmRpDannUADIEeXZlxptqSiSgUiGYcF0jtNIObE8A0 F8DFOsxcKtXid4zHM6qAWFJra/w9NrBhza3k1icRch8j7fni8fbuilNWXRHPHsGQAxb3 DAgQWg0IIewaqBigpfDt5VPW4zEo1iqsg78PbIZJHctPVdEMLceHFKYvuBR0rY1FVFDk L0LTYCpMwT4mm/DeYSynhn9gHOMdKoDgwTHXuKjpuz99L7rUelFn9gOvPWUA1IXRwngI BIvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=T03oF8duMhtzSYksLQyHmY5K2x9PKW2i5l7rywMOwFo=; b=ikgiCG1/y6agbTBBgd3LbAsuOeot2Grle3z69SzpuQNpUhA/i4IoF6UaGyTWb9gGfI 5lAXw00ToIXIfclZQb7l3I36AzByqBDTf0s8QZEw1PRt8oo3SCKUwX/gcwnx+gferH3j Yyud5DzzFPhEK2tDTyqx3p9BxOKxvSE6qp4cNHTW1MkZOS2DYWgUbeEjkurCM8vujNhf E4NIvwkuUlC5wfaSxjoPoMnpYfcwGo5RHppnziyduPwEocRUMrWatDxGH9XeePUsxm26 W3bapa2w1FUpicf/NykMztO9S7dExa2PvK/s7sLCOxgAP5eSSxJttwmBcmg7XqLBuL2J 22cA== X-Gm-Message-State: AHQUAuZ8sDabGdNGhxg3DYfglnGjGKFuQ7pz/8Y+DRVZncZPmAkV+bRE iTUNJSpMR9Xt0evhy96zyRcvhQ== X-Google-Smtp-Source: AHgI3IbpI2MkI9aEsiEzC0Hf4bJ+//1u4Y7EFISBe4bIys0k70DpC9wCL3js+5PrN4HpwBXsFXJU8A== X-Received: by 2002:a0c:e84f:: with SMTP id l15mr17600434qvo.124.1548872536694; Wed, 30 Jan 2019 10:22:16 -0800 (PST) Received: from cakuba.hsd1.ca.comcast.net ([66.60.152.14]) by smtp.gmail.com with ESMTPSA id v32sm1807218qta.37.2019.01.30.10.22.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 30 Jan 2019 10:22:16 -0800 (PST) Date: Wed, 30 Jan 2019 10:22:08 -0800 From: Jakub Kicinski To: =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= Cc: Magnus Karlsson , Network Development , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , intel-wired-lan , Magnus Karlsson Subject: Re: [Intel-wired-lan] [PATCH net] ixgbe: fix potential RX buffer starvation for AF_XDP Message-ID: <20190130102208.63d602d2@cakuba.hsd1.ca.comcast.net> In-Reply-To: References: <1548770630-16189-1-git-send-email-magnus.karlsson@intel.com> <20190129111411.3a4c3599@cakuba.hsd1.ca.comcast.net> Organization: Netronome Systems, Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, 30 Jan 2019 15:16:15 +0100, Bj=C3=B6rn T=C3=B6pel wrote: > Den ons 30 jan. 2019 kl 10:35 skrev Magnus Karlsson: > > > > On Tue, Jan 29, 2019 at 8:15 PM Jakub Kicinski wrote:=20 > > > > > > I may not understand the problem fully, but isn't it kind of normal > > > that if you create a ring empty you'll never receive packets? And it > > > should be reasonably easy to catch while writing an app from scratch > > > (i.e. it behaves deterministically). =20 > > > > Agree that this should be the normal behavior for a NIC. The question > > is how to get out of this situation.There are two options: punt this > > to the application writer or fix this in the driver. I chose to fix > > the driver since this removes complexity in the application. > > =20 >=20 > Magnus' fix addresses a race/timing issue. At zero-copy initialization > point, if the fill ring was empty, the driver (both i40e and ixgbe) > would stop retrying to "allocate" zero-copy frames from the fill > ring. So, frames would never be received, even if the fill ring was > filled at a later point. >=20 > If the driver runs-dry in terms of Rx buffer if one or more frames has > been received, the driver will retry polling the fill-ring. However at > initialization point, if the fill-ring was empty, the driver would > just give up and never retry. >=20 > As Magnus stated, there is no "notify the kernel that the items has > appeared in the fill ring" (other than a HW mechanism where the tail > pointer is a door bell) on the Rx side, so for the Intel drivers it's > up to the driver to solve this. Oh, I see, so its a missing piece in an otherwise very fault tolerant implementation :) Okay, no objection. The nfp prototype I did simply fails the program load if there are not enough buffers to do the initial fill, but that's just a personal preference. I tend to return errors in unclear situations more than most people.